Image coding method, image coding apparatus, image decoding method, image decoding apparatus, and image coding and decoding apparatus

ABSTRACT

An image coding method for coding an image on a block-by-block basis to generate a bitstream includes: deriving, as a first merging candidate, a merging candidate which is a combination of a prediction direction, a motion vector, and a reference picture index for use in coding of the current block; deriving, as a second merging candidate, a merging candidate having a motion vector which is a predetermined vector; selecting a merging candidate to be used for the coding of the current block from the derived first merging candidate and the derived second merging candidate; and attaching an index for identifying the selected merging candidate to the bitstream.

CROSS REFERENCE TO RELATED APPLICATION

The present application claims the benefit of U.S. Provisional PatentApplication No. 61/490,777 filed May 27, 2011. The entire disclosure ofthe above-identified application, including the specification, drawingsand claims are incorporated herein by reference in their entirety.

TECHNICAL FIELD

The present disclosure relates to an image coding method and an imagedecoding method.

BACKGROUND ART

Generally, in coding processing of a moving picture, the amount ofinformation is reduced by compression for which redundancy of a movingpicture in spatial direction and temporal direction is made use of.Generally, conversion to a frequency domain is performed as a method inwhich redundancy in spatial direction is made use of, and coding usingprediction between pictures (the prediction is hereinafter referred toas inter prediction) is performed as a method of compression for whichredundancy in temporal direction is made use of. In the inter predictioncoding, a current picture is coded using, as a reference picture, acoded picture which precedes or follows the current picture in order ofdisplay time. Subsequently, a motion vector is derived by performingmotion estimation on the current picture with reference to the referencepicture. Then, redundancy in temporal direction is removed using acalculated difference between picture data of the current picture andprediction picture data which is obtained by motion compensation basedon the derived motion vector (see Non-patent Literature 1, for example).Here, in the motion estimation, difference values between current blocksin the current picture and blocks in the reference picture arecalculated, and a block having the smallest difference value in thereference picture is determined as a reference block. Then, a motionvector is estimated from the current block and the reference block.

CITATION LIST Non Patent Literature

-   [Non-patent Literature 1]ITU-T Recommendation H.264 “Advanced video    coding for generic audiovisual services”, March 2010-   [Non-patent Literature 2]JCT-VC, “WD3: Working Draft 3 of    High-Efficiency Video Coding”, JCTVC-E603, March 2011

SUMMARY OF INVENTION Technical Problem

It is still desirable to increase coding efficiency for image coding anddecoding in which inter prediction is used, beyond the above-describedconventional technique.

In view of this, the object of the present disclosure is to provide animage coding method and an image decoding method with which codingefficiency for image coding and image decoding using inter prediction isincreased.

Solution to Problem

An image coding method according to an aspect of the present disclosureis an image coding method for coding an image on a block-by-block basisto generate a bitstream, and includes: deriving, as a first mergingcandidate, a merging candidate based on a prediction direction, a motionvector, and a reference picture index which have been used for coding ablock spatially or temporally neighboring a current block to be coded,the merging candidate being a combination of a prediction direction, amotion vector, and a reference picture index for use in coding of thecurrent block; deriving, as a second merging candidate, a mergingcandidate having a motion vector which is a predetermined vector;selecting a merging candidate to be used for the coding of the currentblock from the derived first merging candidate and the derived secondmerging candidate; and attaching an index for identifying the selectedmerging candidate to the bitstream.

It should be noted that these general or specific aspects can beimplemented as a system, a method, an integrated circuit, a computerprogram, a computer-readable recording medium such as a compact discread-only memory (CD-ROM), or as any combination of a system, a method,an integrated circuit, a computer program, and a computer-readablerecording medium.

Advantageous Effects of Invention

According to an aspect of the present disclosure, coding efficiency forimage coding and decoding using inter prediction can be increased.

BRIEF DESCRIPTION OF DRAWINGS

These and other objects, advantages and features of the invention willbecome apparent from the following description thereof taken inconjunction with the accompanying drawings that illustrate a specificembodiment of the present invention. In the Drawings:

FIG. 1A is a diagram for illustrating an exemplary reference picturelist for a B picture;

FIG. 1B is a diagram for illustrating an exemplary reference picturelist of a prediction direction 0 for a B picture;

FIG. 1C is a diagram for illustrating an exemplary reference picturelist of a prediction direction 1 for a B picture;

FIG. 2 is a diagram for illustrating motion vectors for use in thetemporal motion vector prediction mode;

FIG. 3 shows an exemplary motion vector of a neighboring block for usein the merging mode;

FIG. 4 is a diagram for illustrating an exemplary merging blockcandidate list;

FIG. 5 shows a relationship between the size of a merging blockcandidate list and bit sequences assigned to merging block candidateindexes;

FIG. 6 is a flowchart showing an example of a process for coding whenthe merging mode is used;

FIG. 7 is a flowchart showing a process for decoding using the mergingmode;

FIG. 8 shows syntax for attachment of merging block candidate indexes toa bitstream;

FIG. 9 is a block diagram showing a configuration of an image codingapparatus according to Embodiment 1;

FIG. 10 is a flowchart showing processing operations of the image codingapparatus according to Embodiment 1;

FIG. 11 shows an exemplary merging block candidate list according toEmbodiment 1;

FIG. 12 is a flowchart illustrating a process for calculating mergingblock candidates and the size of a merging block candidate listaccording to Embodiment 1;

FIG. 13 is a flowchart illustrating a process for determining whether ornot a merging block candidate is a usable-for-merging candidate andupdating the total number of usable-for-merging candidates according toEmbodiment 1;

FIG. 14 is a flowchart illustrating a process for adding a zero mergingblock candidate according to Embodiment 1;

FIG. 15 is a flowchart illustrating a process for determining whether ornot there is a zero merging block candidate according to Embodiment 1;

FIG. 16 shows an example of a zero merging block according to Embodiment1;

FIG. 17 is a flowchart illustrating a process for selecting a mergingblock candidate according to Embodiment 1;

FIG. 18 is a block diagram showing a configuration of an image codingapparatus according to Embodiment 2;

FIG. 19 is a flowchart showing processing operations of the image codingapparatus according to Embodiment 2;

FIG. 20 is a block diagram showing a configuration of an image decodingapparatus according to Embodiment 3;

FIG. 21 is a flowchart showing processing operations of the imagedecoding apparatus according to Embodiment 3;

FIG. 22 is a block diagram showing a configuration of an image decodingapparatus according to Embodiment 4;

FIG. 23 is a flowchart showing processing operations of the imagedecoding apparatus according to Embodiment 4;

FIG. 24 is a block diagram showing a configuration of an image codingapparatus according to Embodiment 5;

FIG. 25 is a flowchart showing processing operations of the image codingapparatus according to Embodiment 5;

FIG. 26 shows an exemplary merging block candidate list according toEmbodiment 5;

FIG. 27 is a flowchart illustrating a process for calculating mergingblock candidates and the size of a merging block candidate listaccording to Embodiment 5;

FIG. 28 is a flowchart illustrating a process for updating a totalnumber of usable-for-merging candidates according to Embodiment 5,

FIG. 29 is a flowchart illustrating a process for adding a new candidateaccording to Embodiment 5;

FIG. 30 is a block diagram showing a configuration of an image codingapparatus according to Embodiment 6;

FIG. 31 is a flowchart showing processing operations of the image codingapparatus according to Embodiment 6;

FIG. 32 is a block diagram showing a configuration of an image decodingapparatus according to Embodiment 7;

FIG. 33 is a flowchart showing processing operations of the imagedecoding apparatus according to Embodiment 7;

FIG. 34 is a flowchart illustrating a process for setting the size of amerging block candidate list according to Embodiment 7;

FIG. 35 is a flowchart illustrating a process for calculating a mergingblock candidate according to Embodiment 7;

FIG. 36 shows syntax for attachment of merging block candidate indexesto a bitstream;

FIG. 37 shows exemplary syntax in the case where the size of a mergingblock candidate list is fixed at the maximum value of the total numberof merging block candidates;

FIG. 38 is a block diagram showing a configuration of an image decodingapparatus according to Embodiment 8;

FIG. 39 is a flowchart showing processing operations of the imagedecoding apparatus according to Embodiment 8;

FIG. 40 shows an overall configuration of a content providing system forimplementing content distribution services;

FIG. 41 shows an overall configuration of a digital broadcasting system;

FIG. 42 shows a block diagram illustrating an example of a configurationof a television;

FIG. 43 is a block diagram illustrating an example of a configuration ofan information reproducing/recording unit that reads and writesinformation from and on a recording medium that is an optical disk;

FIG. 44 shows an example of a configuration of a recording medium thatis an optical disk;

FIG. 45A shows an example of a cellular phone;

FIG. 45B is a block diagram showing an example of a configuration of acellular phone;

FIG. 46 illustrates a structure of multiplexed data;

FIG. 47 schematically shows how each stream is multiplexed inmultiplexed data;

FIG. 48 shows how a video stream is stored in a stream of PES packets inmore detail;

FIG. 49 shows a structure of TS packets and source packets in themultiplexed data;

FIG. 50 shows a data structure of a PMT;

FIG. 51 shows an internal structure of multiplexed data information;

FIG. 52 shows an internal structure of stream attribute information;

FIG. 53 shows steps for identifying video data;

FIG. 54 is a block diagram showing an example of a configuration of anintegrated circuit for implementing the moving picture coding method andthe moving picture decoding method according to each of embodiments;

FIG. 55 shows a configuration for switching between driving frequencies;

FIG. 56 shows steps for identifying video data and switching betweendriving frequencies;

FIG. 57 shows an example of a look-up table in which video datastandards are associated with driving frequencies;

FIG. 58A is a diagram showing an example of a configuration for sharinga module of a signal processing unit; and

FIG. 58B is a diagram showing another example of a configuration forsharing a module of the signal processing unit.

DESCRIPTION OF EMBODIMENTS

(Underlying Knowledge Forming Basis of the Present Disclosure)

In a moving picture coding scheme already standardized, which isreferred to as H.264, three picture types of I picture, P picture, and Bpicture are used for reduction of the amount of information bycompression.

The I picture is not coded by inter prediction coding. Specifically, theI picture is coded by prediction within the picture (the prediction ishereinafter referred to as intra prediction). The P picture is coded byinter prediction coding with reference to one coded picture preceding orfollowing the current picture in order of display time. The B picture iscoded by inter prediction coding with reference to two coded picturespreceding and following the current picture in order of display time.

In inter prediction coding, a reference picture list for identifying areference picture is generated. In a reference picture list, referencepicture indexes are assigned to coded reference pictures to bereferenced in inter prediction. For example, two reference picture lists(L0, L1) are generated for a B picture because it can be coded withreference to two pictures.

FIG. 1A is a diagram for illustrating an exemplary reference picturelist for a B picture. FIG. 1B shows an exemplary reference picture list0 (L0) for a prediction direction 0 in bi-directional prediction. In thereference picture list 0, the reference picture index 0 having a valueof 0 is assigned to a reference picture 0 with a display order number 2.The reference picture index 0 having a value of 1 is assigned to areference picture 1 with a display order number 1. The reference pictureindex 0 having a value of 2 is assigned to a reference picture 2 with adisplay order number 0. In other words, the shorter the temporaldistance of a reference picture from the current picture, the smallerthe reference picture index assigned to the reference picture.

On the other hand, FIG. 1C shows an exemplary reference picture list 1(L1) for a prediction direction 1 in bi-directional prediction. In thereference picture list 1, the reference picture index 1 having a valueof 0 is assigned to a reference picture 1 with a display order number 1.The reference picture index 1 having a value of 1 is assigned to areference picture 0 with a display order number 2. The reference pictureindex 2 having a value of 2 is assigned to a reference picture 2 with adisplay order number 0.

In this manner, it is possible to assign reference picture indexeshaving values different between prediction directions to a referencepicture (the reference pictures 0 and 1 in FIG. 1A) or to assign thereference picture index having the same value for both directions to areference picture (the reference picture 2 in FIG. 1A).

In a moving picture coding method referred to as H.264 (see Non-patentLiterature 1), a motion vector estimation mode is available as a codingmode for inter prediction of each current block in a B picture. In themotion vector estimation mode, a difference value between picture dataof a current block and prediction picture data and a motion vector usedfor generating the prediction picture data are coded. In addition, inthe motion vector estimation mode, bi-directional prediction anduni-directional prediction can be selectively performed. Inbi-directional prediction, a prediction picture is generated withreference to two coded pictures one of which precedes a current pictureto be coded and the other of which follows the current picture. Inuni-directional prediction, a prediction picture is generated withreference to one coded picture preceding or following a current pictureto be coded.

Furthermore, in the moving picture coding method referred to as H.264, acoding mode referred to as a temporal motion vector prediction mode canbe selected for derivation of a motion vector in coding of a B picture.The inter prediction coding method performed in the temporal motionvector prediction mode will be described below using FIG. 2. FIG. 2 is adiagram for illustrating motion vectors for use in the temporal motionvector prediction mode. Specifically, FIG. 2 shows a case where a blocka in a picture B2 is coded in temporal motion vector prediction mode.

In the coding, a motion vector vb is used which has been used for codinga block b located in the same position in a picture P3, which is areference picture following the picture B2, as the position of the blocka in the picture B2 (in the case, the block b is hereinafter referred toas a co-located block of the block a). The motion vector vb is a motionvector used for coding the block b with reference to the picture P1.

Two reference blocks for the block a are obtained from a forwardreference picture and a backward reference picture, that is, a pictureP1 and a picture P3 using motion vectors parallel to the motion vectorvb. Then, the block a is coded by bi-directional prediction based on thetwo obtained reference blocks. Specifically, in the coding of the blocka, a motion vector va1 is used to reference the picture P1, and a motionvector va2 is used to reference the picture P3.

In addition, a merging mode is discussed as an inter prediction mode forcoding of each current block in a B picture or a P picture (seeNon-patent Literature 2). In the merging mode, a current block is codedusing a prediction direction, a motion vector, and a reference pictureindex which are duplications of those used for coding a neighboringblock of the current block. At this time, the duplications of the indexand others of the neighboring block are attached to a bitstream so thatthe motion direction, motion vector, and reference picture index usedfor the coding can be selected in decoding. A concrete example for it isgiven below with reference to FIG. 3.

FIG. 3 shows an exemplary motion vector of a neighboring block for usein the merging mode. In FIG. 3, a neighboring block A is a coded blocklocated on the immediate left of a current block. A neighboring block Bis a coded block located immediately above the current block. Aneighboring block C is a coded block located immediately right above thecurrent block. A neighboring block D is a coded block locatedimmediately left below the current block.

The neighboring block A is a block coded by uni-directional predictionin the prediction direction 0. The neighboring block A has a motionvector MvL0_A having the prediction direction 0 as a motion vector withrespect to a reference picture indicated by a reference picture indexRefL0_A of the prediction direction 0. Here, MvL0 indicates a motionvector which references a reference picture specified in a referencepicture list 0 (L0). MvL1 indicates a motion vector which references areference picture specified in a reference picture list 1 (L1).

The neighboring block B is a block coded by uni-directional predictionin the prediction direction 1. The neighboring block B has a motionvector MvL1_B having the prediction direction 1 as a motion vector withrespect to a reference picture indicated by a reference picture indexRefL1_B of the prediction direction 1.

The neighboring block C is a block coded by intra prediction.

The neighboring block D is a block coded by uni-directional predictionin the prediction direction 0. The neighboring block D has a motionvector MvL0_D having the prediction direction 0 as a motion vector withrespect to a reference picture indicated by a reference picture indexRefL0_D of the prediction direction 0.

In this case, for example, a combination of a prediction direction, amotion vector, and a reference picture index with which the currentblock can be coded with the highest coding efficiency is selected as aprediction direction, a motion vector, and a reference picture index ofthe current block from the prediction directions, motion vectors andreference picture indexes of the neighboring blocks A to D, and aprediction direction, a motion vector, and a reference picture indexwhich are calculated using a co-located block in temporal motion vectorprediction mode. Then, a merging block candidate index indicating theselected block having the prediction direction, motion vector, andreference picture index is attached to a bitstream.

For example, when the neighboring block A is selected, the current blockis coded using the motion vector MvL0_A having the prediction direction0 and the reference picture index RefL0_A. Then, only the merging blockcandidate index having a value of 0 which indicates use of theneighboring block A as shown in FIG. 4 is attached to a bitstream. Theamount of information on a prediction direction, a motion vector, and areference picture index is thereby reduced.

Furthermore, in the merging mode, a candidate which cannot be used forcoding (hereinafter referred to as an unusable-for-merging candidate),and a candidate having a combination of a prediction direction, a motionvector, and a reference picture index identical to a combination of aprediction direction, a motion vector, and a reference picture index ofany other merging block (hereinafter referred to as an identicalcandidate) are removed from merging block candidates as shown in FIG. 4.

In this manner, the total number of merging block candidates is reducedso that the amount of code assigned to merging block candidate indexescan be reduced. Here, “unusable for merging” means (1) that the mergingblock candidate has been coded by intra prediction, (2) that the mergingblock candidate is outside the boundary of a slice including the currentblock or the boundary of a picture including the current block, or (3)that the merging block candidate is yet to be coded.

In the example shown in FIG. 4, the neighboring block C is a block codedby intra prediction. The merging block candidate having the mergingblock candidate index 3 is therefore an unusable-for-merging candidateand removed from the merging block candidate list. The neighboring blockD is identical in prediction direction, motion vector, and referencepicture index to the neighboring block A. The merging block candidatehaving the merging block candidate index 4 is therefore removed from themerging block candidate list. As a result, the total number of themerging block candidates is finally three, and the size of the mergingblock candidate list is set at three.

Merging block candidate indexes are coded by variable-length coding byassigning bit sequences according to the size of each merging blockcandidate list as shown in FIG. 5. Thus, in the merging mode, the amountof code is reduced by changing bit sequences assigned to merging modeindexes according to the size of each merging block candidate list.

FIG. 6 is a flowchart showing an example of a process for coding whenthe merging mode is used. In Step S1001, motion vectors, referencepicture indexes, and prediction directions of merging block candidatesare obtained from neighboring blocks and a co-located block. In StepS1002, identical candidates and unusable-for-merging candidates areremoved from the merging block candidates. In Step S1003, the totalnumber of the merging block candidates after the removing is set as thesize of the merging block candidate list. In Step S1004, the mergingblock candidate index to be used for coding the current block isdetermined. In Step S1005, the determined merging block candidate indexis coded by performing variable-length coding in bit sequence accordingto the size of the merging block candidate list.

FIG. 7 is a flowchart showing an example of a process for decoding usingthe merging mode. In Step S2001, motion vectors, reference pictureindexes, and prediction directions of merging block candidates areobtained from neighboring blocks and a co-located block. In Step S2002,identical candidates and unusable-for-merging candidates are removedfrom the merging block candidates. In Step S2003, the total number ofthe merging block candidates after the removing is set as the size ofthe merging block candidate list. In Step S2004, the merging blockcandidate index to be used for decoding a current block is decoded froma bitstream using the size of the merging block candidate list. In StepS2005, decoding of a current block is performed by generating aprediction picture using the merging block candidate indicated by thedecoded merging block candidate index.

FIG. 8 shows syntax for attachment of merging block candidate indexes toa bitstream. In FIG. 8, merge_idx represents a merging block candidateindex, and merge_flag represents a merging flag. NumMergeCand representsthe size of a merging block candidate list. NumMergeCand is set at thetotal number of merging block candidates after unusable-for-mergingcandidates and identical candidates are removed from the merging blockcandidates.

Coding or decoding of an image is performed using the merging mode inthe above-described manner.

However, in the merging mode, a motion vector for use in coding acurrent block is calculated from a merging block candidate neighboringthe current block. Then, for example, when a current block to be codedis a stationary region and its neighboring block is a moving-objectregion, coding efficiency may decrease due to lack of increase inaccuracy of prediction in merging mode because motion vectors usable inthe merging mode are affected by the moving-object region.

An image coding method according to an aspect of the present disclosureis an image coding method for coding an image on a block-by-block basisto generate a bitstream, and includes: deriving, as a first mergingcandidate, a merging candidate based on a prediction direction, a motionvector, and a reference picture index which have been used for coding ablock spatially or temporally neighboring a current block to be coded,the merging candidate being a combination of a prediction direction, amotion vector, and a reference picture index for use in coding of thecurrent block; deriving, as a second merging candidate, a mergingcandidate having a motion vector which is a predetermined vector;selecting a merging candidate to be used for the coding of the currentblock from the derived first merging candidate and the derived secondmerging candidate; and attaching an index for identifying the selectedmerging candidate to the bitstream.

With this method, a merging candidate having a motion vector which is apredetermined vector can be derived as a second merging candidate. It istherefore possible to derive a merging candidate having a motion vectorand others for a stationary region as a second merging candidate, forexample. In other words, a current block having a predetermined motionis efficiently coded so that a coding efficiency can be increased.

For example, in the deriving of a merging candidate as a second mergingcandidate, the second merging candidate may be derived for eachreferable reference picture.

With this, a second merging candidate can be derived for each referencepicture. It is therefore possible to increase the variety of mergingcandidates so that coding efficiency can be further increased.

For example, the predetermined vector may be a zero vector.

With this, a merging candidate having a motion vector for a stationaryregion can be derived because the predetermined vector which is a zerovector. It is therefore possible to further increase coding efficiencywhen a current block to be coded is a stationary region.

For example, the image coding method may further include determining amaximum number of merging candidates; and determining whether or not thetotal number of the derived first merging candidate is smaller than themaximum number; wherein the deriving of a merging candidate as a secondmerging candidate is performed when it is determined that the totalnumber of the first merging candidate is smaller than the maximumnumber.

With this, a second merging candidate can be derived when it isdetermined that the total number of the first merging candidates issmaller than the maximum number. Accordingly, the total number ofmerging candidates can be increased within a range not exceeding themaximum number so that coding efficiency can be increased.

For example, in the attaching, the index may be coded using thedetermined maximum number, and the coded index may be attached to thebitstream.

With this method, an index for identifying a merging candidate can becoded using the determined maximum number. In other words, an index canbe coded independently of the total number of actually derived mergingcandidates. Therefore, even when information necessary for derivation ofa merging candidate (for example, information on a co-located block) islost, an index can be still decoded and error resistance is therebyenhanced. Furthermore, an index can be decoded independently of thetotal number of actually derived merging candidates. In other words, anindex can be decoded without waiting for derivation of mergingcandidates. In other words, a bitstream can be generated for whichderiving of merging candidates and decoding of indexes can be performedin parallel.

For example, in the coding, information indicating the determinedmaximum number may be further attached to the bitstream.

With this, information indicating the determined maximum number can beattached to a bitstream. It is therefore possible to switch maximumnumbers by the appropriate unit so that coding efficiency can beincreased.

For example, in the deriving of a first merging candidate, a mergingcandidate which is a combination of a prediction direction, a motionvector, and a reference picture index may be derived as the firstmerging candidate, the combination being different from a combination ofa prediction direction, a motion vector, and a reference picture indexof any first merging candidate previously derived.

With this, a merging candidate which is a combination of a predictiondirection, a motion vector, and a reference picture index identical to acombination of a prediction direction, a motion vector, and a referencepicture index of any first merging candidate previously derived can beremoved from the first merging candidates. As a result, the total numberof the second merging candidates can be increased so that the variety ofcombinations of a prediction direction, a motion vector, and a referencepicture index for a selectable merging candidate can be increased. It istherefore possible to further increase coding efficiency.

For example, the image coding method may further include switching acoding process between a first coding process conforming to a firststandard and a second coding process conforming to a second standard;and attaching, to the bitstream, identification information indicatingeither the first standard or the second standard to which the codingprocess after the switching conforms, wherein when the coding process isswitched to the first coding process, the deriving of a mergingcandidate as a first merging candidate, the deriving of a mergingcandidate as a second merging candidate, the selecting, and theattaching are performed as the first coding process.

With this, it is possible to switchably perform the first coding processconforming to the first standard and the second coding processconforming to the second standard.

Furthermore, an image decoding method according to an aspect of thepresent disclosure is a method for decoding, on a block-by-block basis,a coded image included in a bitstream and includes: deriving, as a firstmerging candidate, a merging candidate based on a prediction direction,a motion vector, and a reference picture index which have been used fordecoding a block spatially or temporally neighboring a current block tobe decoded, the merging candidate being a combination of a predictiondirection, a motion vector, and a reference picture index for use indecoding of the current block; deriving, as a second merging candidate,a merging candidate having a motion vector which is a predeterminedvector; obtaining, from the bitstream, an index for identifying amerging candidate; and selecting, based on the obtained index, a mergingcandidate from the first merging candidate and the second mergingcandidate, the merging candidate being to be used for the decoding of acurrent block.

With this method, a merging candidate having a motion vector which is apredetermined vector can be derived as a second merging candidate. It istherefore possible to derive a merging candidate having a motion vectorand others for a stationary region as a second merging candidate, forexample. This makes it possible to appropriately decode a bitstreamgenerated by coding a block having a predetermined motion efficiently,so that a bitstream coded with increased coding efficiency can beappropriately decoded.

For example, in the deriving of a merging candidate as a second mergingcandidate, the second merging candidate may be derived for eachreferable reference picture.

With this, a second merging candidate can be derived for each referencepicture. The variety of merging candidates is thereby increased, so thata bitstream coded with further increased coding efficiency can beappropriately decoded.

For example, the predetermined vector may be a zero vector.

With this, a merging candidate having a motion vector for a stationaryregion can be derived because the predetermined vector which is a zerovector. It is therefore possible to appropriately decode a bitstreamcoded with increased coding efficiency.

For example, the image decoding method may further include: determininga maximum number of merging candidates; and determining whether or notthe total number of the derived first merging candidate is smaller thanthe maximum number, wherein the deriving of a merging candidate as asecond merging candidate is performed when it is determined that thetotal number of the derived first merging candidate is smaller than themaximum number.

With this, a second merging candidate can be derived when it isdetermined that the total number of the first merging candidates issmaller than the maximum number. Accordingly, the total number ofmerging candidates can be increased within a range not exceeding themaximum number so that a bitstream coded with increased codingefficiency can be appropriately decoded.

For example, in the obtaining, the index coded and attached to thebitstream may be obtained by decoding the index using the determinedmaximum number.

With this, an index for identifying a merging candidate can be decodedusing a determined maximum number. In other words, an index can bedecoded independently of the total number of actually derived mergingcandidates. Therefore, even when information necessary for derivation ofa merging candidate (for example, information on a co-located block) islost, an index can be still decoded and error resistance is therebyenhanced. Furthermore, an index can be decoded without waiting forderivation of merging candidates so that deriving of merging candidatesand decoding of indexes can be performed in parallel.

For example, in the determining of a maximum number of a mergingcandidate, the maximum number may be determined based on informationattached to the bitstream and indicating the maximum number.

With this, a maximum number can be determined based on informationattached to a bitstream. It is therefore possible to decode an imagecoded using maximum numbers changed by the appropriate unit.

For example, in the deriving of a merging candidate as a first mergingcandidate, a merging candidate which is a combination of a predictiondirection, a motion vector, and a reference picture index may be derivedas the first merging candidate, the combination being different from acombination of a prediction direction, a motion vector, and a referencepicture index of any first merging candidate previously derived.

With this, a merging candidate which is a combination of a predictiondirection, a motion vector, and a reference picture index identical to acombination of a prediction direction, a motion vector, and a referencepicture index of any first merging candidate previously derived can beremoved from the first merging candidates. As a result, the total numberof the second merging candidates can be increased so that the variety ofcombinations of a prediction direction, a motion vector, and a referencepicture index for a selectable merging candidate can be increased. It istherefore possible to appropriately decode a bitstream coded withfurther increased coding efficiency.

For example, the image decoding method may further include: switching adecoding process between a first decoding process conforming to a firststandard and a second decoding process conforming to a second standard,according to identification information which is attached to thebitstream and indicates either the first standard or the secondstandard, wherein when the decoding process is switched to the firstdecoding process, the deriving of a merging candidate as a first mergingcandidate, the deriving of a merging candidate as a second mergingcandidate, the obtaining, and the selecting are performed as the firstdecoding process.

With this, it is possible to switchably perform the first decodingprocess conforming to the first standard and the second decoding processconforming to the second standard.

It should be noted that these general or specific aspects can beimplemented as a system, a method, an integrated circuit, a computerprogram, a computer-readable recording medium such as a compact discread-only memory (CD-ROM), or as any combination of a system, a method,an integrated circuit, a computer program, and a computer-readablerecording medium.

An image coding apparatus and an image decoding apparatus according toan aspect of the present disclosure will be described specifically belowwith reference to the drawings.

Each of the exemplary embodiments described below shows a specificexample. The numerical values, shapes, materials, constituent elements,the arrangement and connection of the constituent elements, steps, theprocessing order of the steps etc. shown in the following exemplaryembodiments are mere examples, and therefore do not limit the inventiveconcept in the present disclosure. Furthermore, among the constituentelements in the following exemplary embodiments, constituent elementsnot recited in any one of the independent claims defining the mostgeneric part of the inventive concept are not necessarily required inorder to overcome the disadvantages.

Embodiment 1

FIG. 9 is a block diagram showing a configuration of an image codingapparatus according to Embodiment 1. An image coding apparatus 100 codesan image on a block-by-block basis to generate a bitstream.

As shown in FIG. 9, the image coding apparatus 100 includes a subtractor101, an orthogonal transformation unit 102, a quantization unit 103, aninverse-quantization unit 104, an inverse-orthogonal-transformation unit105, an adder 106, block memory 107, frame memory 108, an intraprediction unit 109, an inter prediction unit 110, an inter predictioncontrol unit 111, a picture-type determination unit 112, a switch 113, amerging block candidate calculation unit 114, colPic memory 115, and avariable-length-coding unit 116.

The subtractor 101 subtracts, on a block-by-block basis, predictionpicture data from input image data included in an input image sequenceto generate prediction error data.

The orthogonal transformation unit 102 transforms the generatedprediction error data from a picture domain into a frequency domain.

The quantization unit 103 quantizes the prediction error datatransformed into a frequency domain.

The inverse-quantization unit 104 inverse-quantizes the prediction errordata quantized by the quantization unit 103.

The inverse-orthogonal-transformation unit 105 transforms theinverse-quantized prediction error data from a frequency domain into apicture domain.

The adder 106 adds, on a block-by-block basis, prediction picture dataand the prediction error data inverse-quantized by theinverse-orthogonal-transformation unit 105 to generate reconstructedimage data.

The block memory 107 stores the reconstructed image data in units of ablock.

The frame memory 108 stores the reconstructed image data in units of aframe.

The picture-type determination unit 112 determines in which of thepicture types of I picture, B picture, and P picture the input imagedata is to be coded. Then, the picture-type determination unit 112generates picture-type information indicating the determined picturetype.

The intra prediction unit 109 generates intra prediction picture data ofa current block by performing intra prediction using reconstructed imagedata stored in the block memory 107 in units of a block.

The inter prediction unit 110 generates inter prediction picture data ofa current block by performing inter prediction using reconstructed imagedata stored in the frame memory 108 in units of a frame and a motionvector derived by a process including motion estimation.

When a current block is coded by intra prediction coding, the switch 113outputs intra prediction picture data generated by the intra predictionunit 109 as prediction picture data of the current block to thesubtractor 101 and the adder 106. On the other hand, when a currentblock is coded by inter prediction coding, the switch 113 outputs interprediction picture data generated by the inter prediction unit 110 asprediction picture data of the current block to the subtractor 101 andthe adder 106.

The merging block candidate calculation unit 114 derives merging blockcandidates for merging mode using motion vectors and others ofneighboring blocks of the current block and a motion vector and othersof the co-located block (colPic information) stored in the colPic memory115. Furthermore, the merging block candidate calculation unit 114 addsthe derived merging block candidate to a merging block candidate list.

Furthermore, the merging block candidate calculation unit 114 derives,as a new candidate, a merging block candidate having a predictiondirection, a motion vector, and a reference picture index for astationary region (hereinafter referred to as a “zero merging blockcandidate”) using a method described later. Then, the merging blockcandidate calculation unit 114 adds the derived zero merging blockcandidate as a new merging block candidate to the merging blockcandidate list. Furthermore, the merging block candidate calculationunit 114 calculates the total number of merging block candidates.

Furthermore, the merging block candidate calculation unit 114 assignsmerging block candidate indexes each having a different value to thederived merging block candidates. Then, the merging block candidatecalculation unit 114 transmits the merging block candidates and mergingblock candidate indexes to the inter prediction control unit 111.Furthermore, the merging block candidate calculation unit 114 transmitsthe calculated total number of merging block candidates to thevariable-length-coding unit 116.

The inter prediction control unit 111 selects a prediction mode usingwhich prediction error is the smaller from a prediction mode in which amotion vector derived by motion estimation is used (motion estimationmode) and a prediction mode in which a motion vector derived from amerging block candidate is used (merging mode). The inter predictioncontrol unit 111 also transmits a merging flag indicating whether or notthe selected prediction mode is the merging mode to thevariable-length-coding unit 116. Furthermore, the inter predictioncontrol unit 111 transmits a merging block candidate index correspondingto the determined merging block candidates to the variable-length-codingunit 116 when the selected prediction mode is the merging mode.Furthermore, the inter prediction control unit 111 transfers the colPicinformation including the motion vector and others of the current blockto the colPic memory 115.

The variable-length-coding unit 116 generates a bitstream by performingvariable-length coding on the quantized prediction error data, themerging flag, and the picture-type information. Thevariable-length-coding unit 116 also sets the total number of mergingblock candidates as the size of the merging block candidate list.Furthermore, the variable-length-coding unit 116 performsvariable-length coding on a merging block candidate index to be used forcoding, by assigning, according to the size of the merging blockcandidate list, a bit sequence to the merging block candidate index.

FIG. 10 is a flowchart showing processing operations of the image codingapparatus 100 according to Embodiment 1.

In Step S101, the merging block candidate calculation unit 114 derivesmerging block candidates from neighboring blocks and a co-located blockof a current block. Furthermore, the merging block candidate calculationunit 114 calculates the size of a merging block candidate list using amethod described later.

For example, in the case shown in FIG. 3, the merging block candidatecalculation unit 114 selects the neighboring blocks A to D as mergingblock candidates. Furthermore, the merging block candidate calculationunit 114 calculates, as a merging block candidate, a co-located mergingblock having a motion vector, a reference picture index, and aprediction direction which are calculated from the motion vector of aco-located block using the time prediction mode.

The merging block candidate calculation unit 114 assigns merging blockcandidate indexes to the respective merging block candidates as shown in(a) in FIG. 11. Next, the merging block candidate calculation unit 114calculates a merging block candidate list as shown in (b) in FIG. 11 andthe size of the merging block candidate list by removingunusable-for-merging candidates and identical candidates and adding anew zero merging block candidate using a method described later.

Shorter codes are assigned to merging block candidate indexes of smallervalues. In other words, the smaller the value of a merging blockcandidate index, the smaller the amount of information necessary forindicating the merging block candidate index.

On the other hand, the larger the value of a merging block candidateindex, the larger the amount of information necessary for the mergingblock candidate index. Therefore, coding efficiency will be increasedwhen merging block candidate indexes of smaller values are assigned tomerging block candidates which are more likely to have motion vectors ofhigher accuracy and reference picture indexes of higher accuracy.

Therefore, a possible case is that the merging block candidatecalculation unit 114 counts the total number of times of selection ofeach merging block candidates as a merging block, and assigns mergingblock candidate indexes of smaller values to blocks with a larger totalnumber of the times. Specifically, this can be achieved by specifying amerging block selected from neighboring blocks and assigning a mergingblock candidate index of a smaller value to the specified merging blockwhen a current block is coded.

When a merging block candidate does not have information such as amotion vector (for example, when the merging block has been a blockcoded by intra prediction, it is located outside the boundary of apicture or the boundary of a slice, or it is yet to be coded), themerging block candidate is unusable for coding.

In Embodiment 1, a merging block candidate unusable for coding isreferred to as an unusable-for-merging candidate, and a merging blockcandidate usable for coding is referred to as a usable-for-mergingcandidate. In addition, among a plurality of merging block candidates, amerging block candidate identical in motion vector, reference pictureindex, and prediction direction to any other merging block is referredto as an identical candidate.

In the case shown in FIG. 3, the neighboring block C is anunusable-for-merging candidate because it is a block coded by intraprediction. The neighboring block D is an identical candidate because itis identical in motion vector, reference picture index, and predictiondirection to the neighboring block A.

In Step S102, the inter prediction control unit 111 selects a predictionmode based on comparison, using a method described later, betweenprediction error of a prediction picture generated using a motion vectorderived by motion estimation and prediction error of a predictionpicture generated using a motion vector obtained from a merging blockcandidate. When the selected prediction mode is the merging mode, theinter prediction control unit 111 sets the merging flag to 1, and whennot, the inter prediction control unit 111 sets the merging flag to 0.

In Step S103, whether or not the merging flag is 1 (that is, whether ornot the selected prediction mode is the merging mode) is determined.

When the result of the determination in Step S103 is true (Yes, S103),the variable-length-coding unit 116 attaches the merging flag to abitstream in Step S104. Subsequently, in Step S105, thevariable-length-coding unit 116 assigns bit sequences according to thesize of the merging block candidate list as shown in FIG. 5 to themerging block candidate indexes of merging block candidates to be usedfor coding. Then, the variable-length-coding unit 116 performsvariable-length coding on the assigned bit sequence.

On the other hand, when the result of the determination in Step S103 isfalse (S103, No), the variable-length-coding unit 116 attachesinformation on a merging flag and a motion estimation vector mode to abitstream in Step S106.

In Embodiment 1, a merging block candidate index having a value of “0”is assigned to the neighboring block A as shown in (a) in FIG. 11. Amerging block candidate index having a value of “1” is assigned to theneighboring block B. A merging block candidate index having a value of“2” is assigned to the co-located merging block. A merging blockcandidate index having a value of “3” is assigned to the neighboringblock C. A merging block candidate index having a value of “4” isassigned to the neighboring block D.

It should be noted that the merging block candidate indexes having sucha value may be assigned otherwise. For example, when a new zero mergingblock candidate is added using a method described later, thevariable-length-coding unit 116 may assign smaller values to preexistentmerging block candidates and a larger value to the new zero mergingblock candidate. In other words, the variable-length-coding unit 116 mayassign a merging block candidate index of a smaller value to apreexistent merging block candidate in priority to a new candidate.

Furthermore, merging block candidates are not limited to the blocks atthe positions of the neighboring blocks A, B, C, and D. For example, aneighboring block located above the lower left neighboring block D canbe used as a merging block candidate. Furthermore, it is not necessaryto use all the neighboring blocks as merging block candidates. Forexample, it is also possible to use only the neighboring blocks A and Bas merging block candidates.

Furthermore, although the variable-length-coding unit 116 attaches amerging block candidate index to a bitstream in Step S105 in FIG. 10 inEmbodiment 1, attaching such a merging block candidate index to abitstream is not always necessary. For example, thevariable-length-coding unit 116 need not attach a merging blockcandidate index to a bitstream when the size of the merging blockcandidate list is “1”. The amount of information on the merging blockcandidate index is thereby reduced.

FIG. 12 is a flowchart showing details of the process in Step S101 inFIG. 10. Specifically, FIG. 12 illustrates a method of calculatingmerging block candidates and the size of a merging block candidate list.FIG. 12 will be described below.

In Step S111, the merging block candidate calculation unit 114determines whether or not a merging block candidate [N] is ausable-for-merging candidate using a method described later.

Here, N denotes an index value for identifying each merging blockcandidate. In Embodiment 1, N takes values from 0 to 4. Specifically,the neighboring block A in FIG. 3 is assigned to a merging blockcandidate [0]. The neighboring block B in FIG. 3 is assigned to amerging block candidate [1]. The co-located merging block is assigned toa merging block candidate [2]. The neighboring block C in FIG. 3 isassigned to a merging block candidate [3]. The neighboring block D inFIG. 3 is assigned to a merging block candidate [4].

In Step S112, the merging block candidate calculation unit 114 obtainsthe motion vector, reference picture index, and prediction direction ofthe merging block candidate [N], and adds them to a merging blockcandidate list.

In Step S113, the merging block candidate calculation unit 114 searchesthe merging block candidate list for an unusable-for-merging candidateand an identical candidate, and removes the unusable-for-mergingcandidate and the identical candidate from the merging block candidatelist as shown in FIG. 11.

In Step S114, the merging block candidate calculation unit 114 adds anew zero merging block candidate to the merging block candidate listusing a method described later. Here, when a new zero merging blockcandidate is added, the merging block candidate calculation unit 114 mayreassign merging block candidate indexes so that the merging blockcandidate indexes of smaller values are assigned to preexistent mergingblock candidates in priority to the new candidate. In other words, themerging block candidate calculation unit 114 may reassign the mergingblock candidate indexes so that a merging block candidate index of alarger value is assigned to the new zero merging block candidate. Theamount of code of merging block candidate indexes is thereby reduced.

In S115, the merging block candidate calculation unit 114 sets the totalnumber of merging block candidates after the adding of the zero mergingblock candidate as the size of the merging block candidate list. In theexample shown in FIG. 11, the total number of merging block candidatescalculated using a method described later is “5”, and the size of themerging block candidate list is set at “5”.

The new zero merging block candidate in Step S114 is a candidate newlyadded to merging block candidates using a method described later whenthe total number of merging block candidates is smaller than a maximumnumber of merging block candidates. In this manner, when the totalnumber of merging block candidates is smaller than a maximum number ofmerging block candidate, the image coding apparatus 100 adds a new zeromerging block candidate so that coding efficiency can be increased.

FIG. 13 is a flowchart showing details of the process in Step S111 inFIG. 12. Specifically, FIG. 13 illustrates a method of determiningwhether or not a merging block candidate [N] is a usable-for-mergingcandidate and updating the total number of usable-for-mergingcandidates. FIG. 13 will be described below.

In Step S121, the merging block candidate calculation unit 114determines whether it is true or false that (1) a merging blockcandidate [N] has been coded by intra prediction, (2) the merging blockcandidate [N] is a block outside the boundary of a slice including thecurrent block or the boundary of a picture including the current block,or (3) the merging block candidate [N] is yet to be coded.

When the result of the determination in Step S121 is true (S121, Yes),the merging block candidate calculation unit 114 sets the merging blockcandidate [N] as an unusable-for-merging candidate in Step S122. On theother hand, when the result of the determination in Step S121 is false(S121, No), the merging block candidate calculation unit 114 sets themerging block candidate [N] as a usable-for-merging candidate in StepS123.

FIG. 14 is a flowchart showing details of the process in Step S114 inFIG. 12. Specifically, FIG. 14 illustrates a method of adding a zeromerging block candidate. FIG. 14 will be described below.

In Step S131, the merging block candidate calculation unit 114determines whether or not the total number of merging block candidatesis smaller than the total number of usable-for-merging candidates. Inother words, the merging block candidate calculation unit 114 determineswhether or not the total number of merging block candidates is stillbelow the maximum number of merging block candidates.

Here, when the result of the determination in Step S131 is true (S131,Yes), in Step S132, the merging block candidate calculation unit 114determines whether or not there is a new zero merging block candidatewhich can be added as a merging block candidate to the merging blockcandidate list. Here, when the result of the determination in Step S132is true (S132, Yes), in Step S133, the merging block candidatecalculation unit 114 assigns a merging block candidate index having avalue to the new zero merging block candidate and adds the new zeromerging block candidate to the merging block candidate list.Furthermore, in Step S134, the merging block candidate calculation unit114 increments the total number of merging block candidates by one.

On the other hand, when the result of the determination in Step S131 orin Step S132 is false (S131 or S132, No), the process for adding a zeromerging block candidate ends. Specifically, when the total number ofmerging block candidates has reached the maximum number of merging blockcandidates, or when there is no new zero merging block candidate, theprocess for adding a new zero merging block candidate ends.

FIG. 15 is a flowchart showing details of the process in Step S132 inFIG. 14. Specifically, FIG. 15 illustrates a method of determiningwhether or not there is a zero merging block candidate. FIG. 15 will bedescribed below.

In Step S141, the merging block candidate calculation unit 114 updatesthe value of a reference picture index refIdxL0 having the predictiondirection 0 and the value of a reference picture index refIdxL1 havingthe prediction direction 1 for use in generating a zero merging blockcandidate. The reference picture indexes refIdxL0 and refIdxL1 each havean initial value of “−1”, and are incremented by “+1” each time theprocess in Step S141 is performed. Specifically, first, the mergingblock candidate calculation unit 114 adds, as a zero merging blockcandidate for a stationary region, a zero merging block candidate whichhas a motion vector being a zero-value vector (zero vector) and areference picture index having a value of 0 to a merging block candidatelist. Next, the merging block candidate calculation unit 114 adds, tothe merging block candidate list, a zero merging block candidate whichhas a motion vector being a zero-value vector and a reference pictureindex having a value of 1.

In Step S142, the merging block candidate calculation unit 114determines whether it is true or false that the updated value of thereference picture index refIdxL0 having the prediction direction 0 issmaller than a maximum number of reference pictures in the referencepicture list 0 for the prediction direction 0, and the updated value ofthe reference picture index refIdxL1 having the prediction direction 1is smaller than a maximum number of reference pictures in the referencepicture list 1 for the prediction direction 1.

Then, when the result of the determination in Step S142 is true (S142,Yes), in Step S143, the merging block candidate calculation unit 114assigns a motion vector which is a zero-value vector (0, 0) and thereference picture index refIdxL0 to the prediction direction 0 of thezero merging block. Furthermore, in Step S144, the merging blockcandidate calculation unit 144 assigns a motion vector which is azero-value vector (0, 0) and the reference picture index refIdxL1 to theprediction direction 1 of the zero merging block.

The merging block candidate calculation unit 114 thus calculates a zeromerging block for bi-directional prediction by the processes in StepS143 and Step S144. FIG. 16 shows an example of a calculated zeromerging block.

In Step S145, the merging block candidate calculation unit 114determines whether or not the merging block candidate list alreadyincludes a merging block candidate identical in motion vector, referencepicture index, and prediction direction to the calculated zero mergingblock candidate. In other words, the merging block candidate calculationunit 114 determines whether or not the calculated zero merging blockcandidate is an identical candidate.

When the result of the determination in S145 is false (S145, No), inStep S146, the merging block candidate calculation unit 114 determinesthat there is a zero merging block candidate.

On the other hand, when the result of the determination in Step S142 isfalse (S142, No) or the result of the determination in Step S145 is true(S145, Yes), in Step S147, the merging block candidate calculation unit114 determines that there is no zero merging block candidate.

In this manner, the merging block candidate calculation unit 114calculates a zero merging block candidate which has a motion vectorbeing a zero-value vector for each referable reference picture, and addsthe calculated zero merging block to a merging block candidate list. Theimage coding apparatus 100 thus can increase efficiency of coding inmerging mode especially when a current block to be coded is a stationaryregion.

FIG. 17 is a flowchart showing details of the process in Step S102 inFIG. 10. Specifically, FIG. 17 illustrates a process for selecting amerging block candidate. FIG. 17 will be described below.

In Step S151, the inter prediction control unit 111 sets a merging blockcandidate index at 0, the minimum prediction error at the predictionerror (cost) in the motion vector estimation mode, and a merging flag at0. Here, the cost is calculated using the following formula for an R-Doptimization model, for example.

Cost=D+λR  (Equation 1)

In Equation 1, D denotes coding distortion. For example, D is the sum ofabsolute differences between original pixel values of a current block tobe coded and pixel values obtained by coding and decoding of the currentblock using a prediction picture generated using a motion vector. Rdenotes the amount of generated codes. For example, R is the amount ofcode necessary for coding a motion vector used for generation of aprediction picture. A denotes an undetermined Lagrange multiplier.

In Step S152, the inter prediction control unit 111 determines whetheror not the value of a merging block candidate index is smaller than thetotal number of merging block candidates of a current block. In otherwords, the inter prediction control unit 111 determines whether or notthere is still a merging block candidate on which the process from StepS153 to Step S155 has not been performed yet.

When the result of the determination in Step S152 is true (S152, Yes),in Step S153, the inter prediction control unit 111 calculates the costfor a merging block candidate to which a merging block candidate indexis assigned. Then, in Step S154, the inter prediction control unit 111determines whether or not the calculated cost for a merging blockcandidate is smaller than the minimum prediction error.

Here, when the result of the determination in Step S154 is true, (S154,Yes), the inter prediction control unit 111 updates the minimumprediction error, the merging block candidate index, and the value ofthe merging flag in Step S155. On the other hand, when the result of thedetermination in Step S154 is false (S154, No), the inter predictioncontrol unit 111 does not update the minimum prediction error, themerging block candidate index, or the value of the merging flag.

In Step S156, the inter prediction control unit 111 increments themerging block candidate index by one, and repeats from Step S152 to StepS156.

On the other hand, when the result of the determination in Step S152 isfalse (S152, No), that is, there is no more unprocessed merging blockcandidate, the inter prediction control unit 111 fixes the final valuesof the merging flag and merging block candidate index in Step S157.

Thus, the image coding apparatus 100 according to Embodiment 1 adds, toa merging block candidate list, a new merging block candidate having amotion vector and a reference picture index for a stationary region sothat coding efficiency can be increased. More specifically, the imagecoding apparatus 100 calculates, for each referable reference picture, amerging block candidate which has a motion vector being a zero-valuevector, and newly adds the calculated merging block candidate to amerging block candidate list, so that efficiency of coding in mergingmode can be increased especially when a current block is a stationaryregion.

It should be noted that the example described in Embodiment 1 in which amerging block candidate which has a motion vector being a zero-valuevector is calculated as a motion vector for a stationary region is notlimiting. For example, with consideration for small camera shake duringvideo shooting, the merging block candidate calculation unit 114 maycalculate a merging block candidate which has, as a motion vector, apredetermined vector having a value slightly larger or smaller than avalue of 0 (for example, a motion vector (0, 1)) as a new candidateinstead of a zero merging block candidate. In this case, thevariable-length-coding unit 116 may add offset parameters, for example,(OffsetX, OffsetY) to a header or others of a sequence, a picture, or aslice. In this case, the merging block candidate calculation unit 114calculates a merging block candidate having a motion vector (OffsetX,OffsetY) as a new candidate.

It should be noted that the example described in Embodiment 1 in whichmerging flag is always attached to a bitstream in merging mode is notlimiting. For example, the merging mode may be forcibly selecteddepending on a block shape for use in inter prediction of a currentblock. In this case, it is possible to reduce the amount of informationby attaching no merging flag to a bitstream.

It should be noted that the example described in Embodiment 1 where themerging mode is used in which a current block is coded using aprediction direction, a motion vector, and a reference picture indexcopied from a neighboring block of the current block is not limiting.For example, a skip merging mode may be used. In the skip merging mode,a current block is coded in the same manner as in the merging mode,using a prediction direction, a motion vector, and a reference pictureindex copied from a neighboring block of the current block withreference to a merging block candidate list created as shown in (b) inFIG. 11. When all resultant prediction errors are zero for the currentblock, a skip flag set at 1 and the skip flag and a merging blockcandidate index are attached to a bitstream. When any of the resultantprediction errors is non-zero, a skip flag is set at 0 and the skipflag, a merging flag, a merging block candidate index, and theprediction errors are attached to a bitstream.

It should be noted that the example described in Embodiment 1 where themerging mode is used in which a current block is coded using aprediction direction, a motion vector, and a reference picture indexcopied from a neighboring block of the current block is not limiting.For example, a motion vector in the motion vector estimation mode may becoded using a merging block candidate list created as shown in (b) inFIG. 11. Specifically, a difference is calculated by subtracting amotion vector of a merging block candidate indicated by a merging blockcandidate index from a motion vector in the motion vector estimationmode. Furthermore, the calculated difference and the merging blockcandidate index may be attached to a bitstream.

Optionally, a difference may be calculated by scaling a motion vectorMV_Merge of a merging block candidate using a reference picture indexRefIdx_ME in the motion estimation mode and a reference picture indexRefIdx_Merge of the merging block candidate and subtracting a motionvector scaledMV_Merge of the merging block candidate after the scalingfrom the motion vector in the motion estimation mode. Then, thecalculated difference and the merging block candidate index may beattached to a bitstream. The following is an exemplary formula for thescaling.

scaledMV_Merge=MV_Mergex(POC(RefIdx_(—)ME)−curPOC)/(POC(RefIdx_Merge)−curPOC)  (Equation 2)

Here, POC (Refldx_ME) denotes the display order of a reference pictureindicated by a reference picture index Refldx_ME. POC (Refldx_Merge)denotes the display order of a reference picture indicated by areference picture index Refldx_Merge. curPOC denotes the display orderof a current picture to be coded.

It should be noted that the example described in Embodiment 1 in which azero merging block candidate for bi-directional prediction is generatedfrom a motion vector being a zero-value vector, a reference pictureindex having a prediction direction 0, and a reference picture indexhaving a prediction direction 1 is not limiting. For example, themerging block candidate calculation unit 114 may generate a zero mergingblock candidate having a prediction direction 0 using a motion vectorbeing a zero-value vector and a reference picture index having aprediction direction 0, and add the zero merging block candidate to amerging block candidate list. Furthermore, similarly, the merging blockcandidate calculation unit 114 may generate a zero merging blockcandidate having a prediction direction 1 using a motion vector being azero-value vector and a reference picture index having a predictiondirection 1, and add the zero merging block candidate to a merging blockcandidate list.

It should be noted that the example described in Embodiment 1 in whichzero merging block candidates are generated using a reference pictureindex having an initial value of 0 and incremented by “+1” is notlimiting. For example, the merging block candidate calculation unit 114may generate zero merging block candidates in order of reference pictureindexes assigned to the reference pictures starting from a referencepicture index assigned to the reference picture closest to a currentpicture to a reference picture index assigned to the farthest one indisplay order.

Embodiment 2

Although the merging block candidate calculation unit 114 determines inStep S145 in FIG. 15 whether or not a zero merging block candidate is anidentical candidate in Embodiment 1, this determination is not alwaysnecessary. For example, the determination in Step S145 may be omitted.This reduces computational complexity in derivation of a merging blockcandidate list for the image coding apparatus 100.

Furthermore, it should be noted that Embodiment 1 in which zero mergingblock candidates are added to a merging block candidate list until thetotal number of merging block candidates reaches a maximum number ofmerging block candidates is not limiting. For example, the merging blockcandidate calculation unit 114 may determine in S131 in FIG. 14 whetheror not the total number of merging block candidates has reached apredetermined threshold value which is smaller than a maximum number ofmerging block candidates. This reduces computational complexity inderivation of a merging block candidate list for the image codingapparatus 100.

Furthermore, it should be noted that Embodiment 1 in which adding a zeromerging block candidate to a merging block candidate list is ended whenthe total number of merging block candidates reaches a maximum number ofmerging block candidates is not limiting. For example, the determinationin Step S131 in FIG. 14 as to whether or not the total number of mergingblock candidates has reached the maximum number of merging blockcandidates may be omitted, and the merging block candidate calculationunit 114 may add all new zero merging block candidates to the mergingblock candidate list until it turns out that there is no more new zeromerging block candidate. This widens the range of optional merging blockcandidates for the image coding apparatus 100 so that coding efficiencycan be increased.

Such a modification of the image coding apparatus according toEmbodiment 1 will be specifically described below as an image codingapparatus according to Embodiment 2.

FIG. 18 is a block diagram showing a configuration of an image codingapparatus 200 according to Embodiment 2. The image coding apparatus 200codes an image on a block-by-block basis to generate a bitstream. Theimage coding apparatus 200 includes a merging candidate derivation unit210, a prediction control unit 220, and a coding unit 230.

The merging candidate derivation unit 210 corresponds to the mergingblock candidate calculation unit 114 in Embodiment 1. The mergingcandidate derivation unit 210 derives merging candidates. The mergingcandidate derivation unit 210 generates a merging candidate list inwhich, for example, indexes each identifying a different derived mergingcandidate (hereinafter referred to as merging candidate indexes) areassociated with the respective derived merging candidates.

The merging candidates are candidates each of which is a combination ofa prediction direction, a motion vector, and a reference picture indexfor use in coding of a current block. Specifically, each of the mergingcandidates is a combination including at least a set of a predictiondirection, a motion vector, and a reference picture index.

The merging candidates correspond to the merging block candidates inEmbodiment 1. The merging candidate list is the same as the mergingblock candidate list.

As shown in FIG. 18, the merging candidate derivation unit 210 includesa first derivation unit 211 and a second derivation unit 212.

The first derivation unit 211 derives first merging candidates based on,for example, a prediction direction, a motion vector, and a referencepicture index which have been used for coding a block spatially ortemporally neighboring the current block. Then, for example, the firstderivation unit 211 registers the first merging candidates derived inthis manner in the merging candidate list in association with therespective merging candidate indexes.

The spatially neighboring block is a block which is within a pictureincluding the current block and neighbors the current block.Specifically, the neighboring blocks A to D shown in FIG. 3 are examplesof the spatially neighboring block.

The temporally neighboring block is a block which is within a picturedifferent from a picture including the current block and corresponds tothe current block. Specifically, a co-located block is an example of thetemporally neighboring block.

It should be noted that the temporally neighboring block need not be ablock located in the same position as the current block (co-locatedblock). For example, the temporally neighboring block may be a blockneighboring the co-located block.

It should be noted that the first derivation unit 211 may derive, as afirst merging candidate, a combination of a prediction direction, amotion vector, and a reference picture index which have been used forcoding blocks which spatially neighbor the current block exceptunusable-for-merging blocks. An unusable-for-merging block is a blockcoded by intra prediction, a block outside the boundary of a sliceincluding the current block or the boundary of a picture including thecurrent block, or a block yet to be coded. With this configuration, thefirst derivation unit 211 can derive first merging candidates fromblocks appropriate for obtaining merging candidates.

The second derivation unit 212 derives, as a second merging candidate, amerging candidate having a motion vector which is a predeterminedvector. Specifically, for example, the second derivation unit 212derives a second merging candidate for each referable reference picture.In this manner, the image coding apparatus 200 can increase the varietyof merging candidates, and thereby further increase coding efficiency.

It should be noted that the second derivation unit 212 need not derive asecond merging candidate for each referable reference picture. Forexample, the second derivation unit 212 may derive second mergingcandidates for a predetermined number of reference pictures.

The predetermined vector may be a zero vector as described inEmbodiment 1. With this, the second derivation unit 212 can derive amerging candidate having a motion vector for a stationary region. Theimage coding apparatus 200 thus can achieve further increased codingefficiency when a current block to be coded is a stationary region. Itshould be noted that the predetermined vector need not be a zero vector.

Then, for example, the second derivation unit 212 registers secondmerging candidates derived in this manner in the merging candidate listeach in association with a different merging candidate index. At thistime, the second derivation unit 212 may register the second mergingcandidates in the merging candidate list so that the merging candidateindexes assigned to the first merging candidates are smaller than themerging candidate indexes assigned to the second merging candidates asin Embodiment 1. With this, the image coding apparatus 200 can reducethe code amount when the first merging candidates are more likely to beselected as a merging candidate to be used for coding than a secondmerging candidate so that coding efficiency can be increased.

The prediction control unit 220 selects a merging candidate to be usedfor coding a current block from the derived first merging candidates andsecond merging candidates. In other words, the prediction control unit220 selects a merging candidate to be used for coding a current blockfrom the merging candidate list.

The coding unit 230 attaches an index for identifying the selectedmerging candidate (merging candidate index) to a bitstream. For example,the coding unit 230 codes a merging candidate index using the sum of thetotal number of the derived first merging candidates and the totalnumber of the derived second merging candidates (total number of mergingcandidates), and attaches the coded merging candidate index to abitstream.

Next, operations of the image coding apparatus 200 in theabove-described configuration will be described below.

FIG. 19 is a flowchart showing processing operations of the image codingapparatus 200 according to Embodiment 2.

First, the first derivation unit 211 derives a first merging candidate(S201). Subsequently, the second derivation unit 212 derives a secondmerging candidate (S202).

Next, the prediction control unit 220 selects a merging candidate to beused for coding a current block from the first merging candidates andthe second merging candidates (S203). For example, the predictioncontrol unit 220 selects a merging candidate for which the costrepresented by Equation 1 is a minimum from the merging candidate listas in Embodiment 1.

Finally, the coding unit 230 attaches a coded index for identifying theselected merging candidate to a bitstream (S204).

In this manner, the image coding apparatus 200 according to Embodiment 2can derive, as a second merging candidate, a merging candidate having amotion vector which is a predetermined vector. It is therefore possiblefor the image coding apparatus 200 to derive a merging candidate havinga motion vector and others for a stationary region as a second mergingcandidate, for example. In other words, it is possible for the imagecoding apparatus 200 to efficiently code a current block having apredetermined motion so that coding efficiency can be increased.

Embodiment 3

FIG. 20 is a block diagram showing a configuration of an image decodingapparatus 300 according to Embodiment 3. The image decoding apparatus300 is an apparatus corresponding to the image coding apparatus 200according to Embodiment 1. Specifically, for example, the image decodingapparatus 300 decodes, on a block-by-block basis, coded images includedin a bitstream generated by the image coding apparatus 100 according toEmbodiment 1.

As shown in FIG. 20, the image decoding apparatus 300 includes avariable-length-decoding unit 301, an inverse-quantization unit 302, aninverse-orthogonal-transformation unit 303, an adder 304, block memory305, frame memory 306, an intra prediction unit 307, an inter predictionunit 308, an inter prediction control unit 309, a switch 310, a mergingblock candidate calculation unit 311, and colPic memory 312.

The variable-length-decoding unit 301 generates picture-typeinformation, a merging flag, and a quantized coefficient by performingvariable-length decoding on an input bitstream. Furthermore, thevariable-length-decoding unit 301 performs variable-length decoding on amerging block candidate index using the total number of merging blockcandidates calculated by the merging block candidate calculation unit311.

The inverse-quantization unit 302 inverse-quantizes the quantizedcoefficient obtained by the variable-length decoding.

The inverse-orthogonal-transformation unit 303 generates predictionerror data by transforming an orthogonal transformation coefficientobtained by the inverse quantization from a frequency domain to apicture domain.

The block memory 305 stores, in units of a block, decoded image datagenerated by adding the prediction error data and prediction picturedata.

The frame memory 306 stores decoded image data in units of a frame.

The intra prediction unit 307 generates prediction picture data of acurrent block to be decoded, by performing intra prediction using thedecoded image data stored in the block memory 305 in units of a block.

The inter prediction unit 308 generates prediction picture data of acurrent block to be decoded, by performing inter prediction using thedecoded image data stored in the frame memory 306 in units of a frame.

When a current block is decoded by intra prediction decoding, the switch310 outputs intra prediction picture data generated by the intraprediction unit 307 as prediction picture data of the current block tothe adder 304. On the other hand, when a current block is decoded byinter prediction decoding, the switch 310 outputs inter predictionpicture data generated by the inter prediction unit 308 as predictionpicture data of the current block to the adder 304.

The merging block candidate calculation unit 311 derives merging blockcandidates from motion vectors and others of neighboring blocks of thecurrent block and a motion vector and others of a co-located block(colPic information) stored in the colPic memory 312. Furthermore, themerging block candidate calculation unit 311 adds the derived mergingblock candidate to a merging block candidate list.

Furthermore, the merging block candidate calculation unit 311 derives,as a zero merging block candidate, a merging block candidate having aprediction direction, a motion vector, and a reference picture index fora stationary region using a method described later. Then, the mergingblock candidate calculation unit 311 adds the derived zero merging blockcandidate as a new merging block candidate to the merging blockcandidate list. Furthermore, the merging block candidate calculationunit 311 calculates the total number of merging block candidates.

Furthermore, the merging block candidate calculation unit 311 assignsmerging block candidate indexes each having a different value to thederived merging block candidates. Then, the merging block candidatecalculation unit 311 transmits the merging block candidates to which themerging block candidate indexes have been assigned to the interprediction control unit 309. Furthermore, the merging block candidatecalculation unit 311 transmits the calculated total number of mergingblock candidates to the variable-length-decoding unit 301.

The inter prediction control unit 309 causes the inter prediction unit308 to generate an inter prediction picture using information on motionvector estimation mode when the merging flag decoded is “0”. On theother hand, when the merging flag is “1”, the inter prediction controlunit 309 determines, based on a decoded merging block candidate index, amotion vector, a reference picture index, and a prediction direction foruse in inter prediction from a plurality of merging block candidates.Then, the inter prediction control unit 309 causes the inter predictionunit 308 to generate an inter prediction picture using the determinedmotion vector, reference picture index, and prediction direction.Furthermore, the inter prediction control unit 309 transfers colPicinformation including the motion vector of the current block to thecolPic memory 312.

Finally, the adder 304 generates decoded image data by adding theprediction picture data and the prediction error data.

FIG. 21 is a flowchart showing processing operations of the imagedecoding apparatus 300 according to Embodiment 3.

In Step S301, the variable-length-decoding unit 301 decodes a mergingflag.

In Step S702, when the merging flag is “1” (S302, Yes), in Step S303,the merging block candidate calculation unit 311 generates a mergingblock candidate in the same manner as in Step S101 in FIG. 10.Furthermore, the merging block candidate calculation unit 311 calculatesthe total number of merging block candidates as the size of a mergingblock candidate list.

In Step S304, the variable-length-decoding unit 301 performsvariable-length decoding on a merging block candidate index from abitstream using the size of the merging block candidate list.

In Step S305, the inter prediction control unit 309 causes the interprediction unit 308 to generate an inter prediction picture using themotion vector, reference picture index, and prediction direction of themerging block candidate indicated by the decoded merging block candidateindex.

When the merging flag is “0” in Step S302 (S302, No), in Step S306, theinter prediction unit 308 generates an inter prediction picture usinginformation on motion vector estimation mode decoded by thevariable-length-decoding unit 301.

Optionally, when the size of a merging block candidate list calculatedin Step S303 is “1”, a merging block candidate index may be estimated tobe “0” without being decoded.

In this manner, the image coding apparatus 300 according to Embodiment 3adds, to a merging block candidate list, a new merging block candidatehaving a motion vector and a reference picture index for a stationaryregion. The image decoding apparatus 300 thus can appropriately decode abitstream coded with increased coding efficiency. More specifically, abitstream coded in merging mode with increased efficiency can beappropriately decoded using a merging block candidate list to which anew merging block candidate which is calculated for each referablereference picture and has a motion vector being a zero-value vector areadded, especially in the case where a current block is a stationaryregion.

Embodiment 4

Although the image decoding apparatus according to Embodiment 3 includesconstituent elements as shown in FIG. 20, the image decoding apparatusneed not include all of the constituent elements. Such a modification ofthe image decoding apparatus according to Embodiment 3 will bespecifically described below as an image decoding apparatus according toEmbodiment 4.

FIG. 22 is a block diagram showing a configuration of an image decodingapparatus 400 according to Embodiment 4. The image decoding apparatus400 is an apparatus corresponding to the image coding apparatus 200according to Embodiment 2. For example, the image decoding apparatus 400decodes, on a block-by-block basis, coded images included in a bitstreamgenerated by the image coding apparatus 200 according to Embodiment 2.

As shown in FIG. 22, the image decoding apparatus 400 includes a mergingcandidate derivation unit 410, a decoding unit 420, and a predictioncontrol unit 430.

The merging candidate derivation unit 410 corresponds to the mergingblock candidate calculation unit 311 in Embodiment 3. The mergingcandidate derivation unit 410 derives merging candidates. The mergingcandidate derivation unit 410 generates a merging candidate list inwhich, for example, indexes each identifying a different derived mergingcandidate (merging candidate indexes) are associated with the respectivederived merging candidates.

As shown in FIG. 22, the merging candidate derivation unit 410 includesa first derivation unit 411 and a second derivation unit 412.

The first derivation unit 411 derives first merging candidates in thesame manner as the first derivation unit 211 in Embodiment 2.Specifically, the first derivation unit 411 derives first mergingcandidates based on, for example, a prediction direction, a motionvector, and a reference picture index which have been used for decodinga block spatially or temporally neighboring a current block to bedecoded. Then, for example, the first derivation unit 411 registers thefirst merging candidates derived in this manner in the merging candidatelist in association with the respective merging candidate indexes.

The second derivation unit 412 derives, as a second merging candidate, amerging candidate having a motion vector which is a predeterminedvector. Specifically, the second derivation unit 412 derives a secondmerging candidate in the same manner as the second derivation unit 212in Embodiment 2. Then, for example, the second derivation unit 412registers second merging candidates derived in this manner in themerging candidate list each in association with a different mergingcandidate index.

More specifically, for example, the second derivation unit 412 derives asecond merging candidate for each referable reference picture. Thevariety of merging candidates is thereby increased, so that a bitstreamcoded with further increased coding efficiency can be appropriatelydecoded.

The predetermined vector may be a zero vector as described inEmbodiment 1. The second derivation unit 214 thus can derive a mergingcandidate having a motion vector for a stationary region. With this, theimage decoding apparatus 400 can appropriately decode a bitstream codedwith increased coding efficiency.

The decoding unit 420 obtains an index for identifying a mergingcandidate from a bitstream. For example, the decoding unit 420 obtains amerging candidate index by decoding a coded merging candidate indexattached to a bitstream using the sum of the total number of the derivedfirst merging candidates and the total number of the derived secondmerging candidates (total number of merging candidates).

The prediction control unit 430 selects, based on the obtained index, amerging candidate to be used for decoding a current block from thederived first merging candidates and second merging candidates. In otherwords, the prediction control unit 430 selects a merging candidate to beused for decoding a current block from the merging candidate list.

Next, operations of the image decoding apparatus 400 in theabove-described configuration will be explained below.

FIG. 23 is a flowchart showing processing operations of the imagedecoding apparatus 400 according to Embodiment 4.

First, the first derivation unit 411 derives a first merging candidate(S401). Subsequently, the second derivation unit 412 derives a secondmerging candidate (S402). Then, the decoding unit 420 obtains a mergingcandidate index from a bitstream (S403).

Finally, the prediction control unit 430 selects, based on the obtainedindex, a merging candidate to be used for decoding a current block fromthe first merging candidates and second merging candidates (S404).

In this manner, the image decoding apparatus 400 according to Embodiment4 can derive, as a second merging candidate, a merging candidate havinga motion vector which is a predetermined vector. It is thereforepossible for the image decoding apparatus 400 to derive a mergingcandidate having a motion vector and others for a stationary region as asecond merging candidate, for example. In other words, it is possiblefor the image decoding apparatus 400 to efficiently decode a currentblock having a predetermined motion, and thereby appropriately decode abitstream coded with increased coding efficiency.

Embodiment 5

The merging candidates correspond to the merging block candidates inEmbodiment 1. The method of deriving the size of a merging blockcandidate list according to Embodiment 5 will be described below indetail.

In the merging mode according to Embodiment 1, the total number ofmerging block candidates is set as the size of a merging block candidatelist for use in coding or decoding of a merging block candidate index.The total number of merging block candidates is determined afterunusable-for-merging candidates or identical candidates are removedbased on information on reference pictures including a co-located block.

A discrepancy in bit sequence assigned to a merging block candidateindex is therefore caused between an image coding apparatus and an imagedecoding apparatus in the case where there is a difference in the totalnumber of merging block candidates between the image coding apparatusand the image decoding apparatus. As a result, the image decodingapparatus cannot decode a bitstream correctly.

For example, when information on a reference picture referenced as aco-located block is lost due to packet loss in a transmission path, themotion vector or the reference picture index of the co-located blockbecomes unknown. Accordingly, the information on a merging blockcandidate to be generated from the co-located block becomes unknown. Insuch a case, it is no longer possible to correctly removeunusable-for-merging candidates or identical candidates from the mergingblock candidates in decoding. As a result, the image decoding apparatusfails to obtain the correct size of a merging block candidate list, andit is therefore impossible to normally decode a merging block candidateindex.

Thus, the image coding apparatus according to Embodiment 5 is capable ofcalculating the size of a merging block candidate list for use in codingor decoding of a merging block candidate index, using a methodindependent of information on reference pictures including a co-locatedblock. The image coding apparatus thereby achieves enhanced errorresistance.

FIG. 24 is a block diagram showing a configuration of an image codingapparatus 500 according to Embodiment 5. For FIG. 24, the constituentelements in common with FIG. 9 are denoted with the same referencesigns, and description thereof is omitted.

As shown in FIG. 24, the image coding apparatus 500 includes asubtractor 101, an orthogonal transformation unit 102, a quantizationunit 103, an inverse-quantization unit 104, aninverse-orthogonal-transformation unit 105, an adder 106, block memory107, frame memory 108, an intra prediction unit 109, an inter predictionunit 110, an inter prediction control unit 111, a picture-typedetermination unit 112, a switch 113, a merging block candidatecalculation unit 514, colPic memory 115, and a variable-length-codingunit 516.

The merging block candidate calculation unit 514 derives merging blockcandidates for merging mode using motion vectors and others ofneighboring blocks of the current block and a motion vector and othersof the co-located block (colPic information) stored in the colPic memory115. Then, the merging block candidate calculation unit 514 calculatesthe total number of usable-for-merging candidates using a methoddescribed later.

Furthermore, the merging block candidate calculation unit 514 assignsmerging block candidate indexes each having a different value to thederived merging block candidates. Then, the merging block candidatecalculation unit 514 transmits the merging block candidates and mergingblock candidate indexes to the inter prediction control unit 111.Furthermore, the merging block candidate calculation unit 514 transmitsthe calculated total number of usable-for-merging candidates to thevariable-length-coding unit 116.

The variable-length-coding unit 516 generates a bitstream by performingvariable-length coding on the quantized prediction error data, themerging flag, and the picture-type information. Thevariable-length-coding unit 516 also sets the total number ofusable-for-merging candidates as the size of the merging block candidatelist. Furthermore, the variable-length-coding unit 516 performsvariable-length coding on a merging block candidate index to be used forcoding, by assigning, according to the size of the merging blockcandidate list, a bit sequence to the merging block candidate index.

FIG. 25 is a flowchart showing processing operations of the image codingapparatus 500 according to Embodiment 5. For FIG. 25, the steps incommon with FIG. 10 are denoted with the same reference signs, anddescription thereof is omitted as appropriate.

In Step S501, the merging block candidate calculation unit 514 derivesmerging block candidates from neighboring blocks and a co-located blockof a current block. Furthermore, the merging block candidate calculationunit 514 calculates the size of a merging block candidate list using amethod described later.

For example, in the case shown in FIG. 3, the merging block candidatecalculation unit 514 selects the neighboring blocks A to D as mergingblock candidates. Furthermore, the merging block candidate calculationunit 514 calculates, as a merging block candidate, a co-located mergingblock having a motion vector and others which are calculated from themotion vector of a co-located block using the time prediction mode.

The merging block candidate calculation unit 514 assigns merging blockcandidate indexes to the respective merging block candidates as shown in(a) in FIG. 26. Next, the merging block candidate calculation unit 514calculates a merging block candidate list as shown in (b) in FIG. 26 andthe size of the merging block candidate list by removingunusable-for-merging candidates and identical candidates and adding newcandidates using a method described later.

Shorter codes are assigned to merging block candidate indexes of smallervalues. In other words, the smaller the value of a merging blockcandidate index, the smaller the amount of information necessary forindicating the merging block candidate index.

On the other hand, the larger the value of a merging block candidateindex, the larger the amount of information necessary for the mergingblock candidate index. Therefore, coding efficiency will be increasedwhen merging block candidate indexes of smaller values are assigned tomerging block candidates which are more likely to have motion vectors ofhigher accuracy and reference picture indexes of higher accuracy.

Therefore, a possible case is that the merging block candidatecalculation unit 514 counts the total number of times of selection ofeach merging block candidates as a merging block, and assigns mergingblock candidate indexes of smaller values to blocks with a larger totalnumber of the times. Specifically, this can be achieved by specifying amerging block selected from neighboring blocks and assigning a mergingblock candidate index of a smaller value to the specified merging blockwhen a current block is coded.

When a merging block candidate does not have information such as amotion vector (for example, when the merging block has been a blockcoded by intra prediction, it is located outside the boundary of apicture or the boundary of a slice, or it is yet to be coded), themerging block candidate is unusable for coding.

In Embodiment 5, a merging block candidate unusable for coding isreferred to as an unusable-for-merging candidate, and a merging blockcandidate usable for coding is referred to as a usable-for-mergingcandidate. In addition, among a plurality of merging block candidates, amerging block candidate identical in motion vector, reference pictureindex, and prediction direction to any other merging block is referredto as an identical candidate.

In the case shown in FIG. 3, the neighboring block C is anunusable-for-merging candidate because it is a block coded by intraprediction. The neighboring block D is an identical candidate because itis identical in motion vector, reference picture index, and predictiondirection to the neighboring block A.

In Step S102, the inter prediction control unit 111 selects a predictionmode based on comparison between prediction error of a predictionpicture generated using a motion vector derived by motion estimation andprediction error of a prediction picture generated using a motion vectorobtained from a merging block candidate. When the selected predictionmode is the merging mode, the inter prediction control unit 111 sets themerging flag to 1, and when not, the inter prediction control unit 111sets the merging flag to 0.

In Step S103, whether or not the merging flag is 1 (that is, whether ornot the selected prediction mode is the merging mode) is determined.

When the result of the determination in Step S103 is true (Yes, S103),the variable-length-coding unit 516 attaches the merging flag to abitstream in Step S104. Subsequently, in Step S505, thevariable-length-coding unit 516 assigns bit sequences according to thesize of the merging block candidate list as shown in FIG. 5 to themerging block candidate indexes of merging block candidates to be usedfor coding. Then, the variable-length-coding unit 516 performsvariable-length coding on the assigned bit sequence.

On the other hand, when the result of the determination in Step S103 isfalse (S103, No), the variable-length-coding unit 516 attachesinformation on a merging flag and a motion estimation vector mode to abitstream in Step S106.

In Embodiment 5, a merging block candidate index having a value of “0”is assigned to the neighboring block A as shown in (a) in FIG. 26. Amerging block candidate index having a value of “1” is assigned to theneighboring block B. A merging block candidate index having a value of“2” is assigned to the co-located merging block. A merging blockcandidate index having a value of “3” is assigned to the neighboringblock C. A merging block candidate index having a value of “4” isassigned to the neighboring block D.

It should be noted that the merging block candidate indexes having sucha value may be assigned otherwise. For example, when a new candidate isadded using the method described in Embodiment 1 or a method describedlater, the variable-length-coding unit 516 may assign smaller values topreexistent merging block candidates and a larger value to the newcandidate. In other words, the variable-length-coding unit 516 mayassign a merging block candidate index of a smaller value to apreexistent merging block candidate in priority to a new candidate.

Furthermore, merging block candidates are not limited to the blocks atthe positions of the neighboring blocks A, B, C, and D. For example, aneighboring block located above the lower left neighboring block D canbe used as a merging block candidate. Furthermore, it is not necessaryto use all the neighboring blocks as merging block candidates. Forexample, it is also possible to use only the neighboring blocks A and Bas merging block candidates.

Furthermore, although the variable-length-coding unit 516 attaches amerging block candidate index to a bitstream in Step S505 in FIG. 25 inEmbodiment 5, attaching such a merging block candidate index to abitstream is not always necessary. For example, thevariable-length-coding unit 116 need not attach a merging blockcandidate index to a bitstream when the size of the merging blockcandidate list is “1”. The amount of information on the merging blockcandidate index is thereby reduced.

FIG. 27 is a flowchart showing details of the process in Step S501 inFIG. 25. Specifically, FIG. 27 illustrates a method of calculatingmerging block candidates and the size of a merging block candidate list.FIG. 27 will be described below.

In Step S511, the merging block candidate calculation unit 514determines whether or not a merging block candidate [N] is ausable-for-merging candidate using a method described later. Then, themerging block candidate calculation unit 514 updates the total number ofusable-for-merging candidates according to the result of thedetermination.

Here, N denotes an index value for identifying each merging blockcandidate. In Embodiment 5, N takes values from 0 to 4. Specifically,the neighboring block A in FIG. 3 is assigned to a merging blockcandidate [0]. The neighboring block B in FIG. 3 is assigned to amerging block candidate [1]. The co-located merging block is assigned toa merging block candidate [2]. The neighboring block C in FIG. 3 isassigned to a merging block candidate [3]. The neighboring block D inFIG. 5 is assigned to a merging block candidate [4].

In Step S512, the merging block candidate calculation unit 514 obtainsthe motion vector, reference picture index, and prediction direction ofthe merging block candidate [N], and adds them to a merging blockcandidate list.

In Step S513, the merging block candidate calculation unit 514 searchesthe merging block candidate list for an unusable-for-merging candidateand an identical candidate, and removes the unusable-for-mergingcandidate and the identical candidate from the merging block candidatelist as shown in FIG. 26.

In Step S514, the merging block candidate calculation unit 514 adds anew candidate to the merging block candidate list using a methoddescribed in Embodiment 1 or a method described later. Here, when a newcandidate is added, the merging block candidate calculation unit 514 mayreassign merging block candidate indexes so that the merging blockcandidate indexes of smaller values are assigned to preexistent mergingblock candidates in priority to the new candidate. In other words, themerging block candidate calculation unit 514 may reassign the mergingblock candidate indexes so that a merging block candidate index of alarger value is assigned to the new candidate. The amount of code ofmerging block candidate indexes is thereby reduced.

In Step S515, the merging block candidate calculation unit 514 sets thetotal number of usable-for-merging candidates calculated in Step S511 asthe size of the merging block candidate list. In the example shown inFIG. 26, the calculated number of usable-for-merging candidates is “4”,and the size of the merging block candidate list is set at “4”.

The new candidate in Step S514 is a candidate newly added to mergingblock candidates using the method described in Embodiment 1 or a methoddescribed later when the total number of merging block candidates issmaller than the total number of usable-for-merging candidates. The newcandidate may be a merging block candidate having a motion vector whichis a predetermined vector (for example, a zero vector). Examples of sucha new candidate include a neighboring block located above the lower-leftneighboring block D in FIG. 3, a block corresponding to any ofneighboring blocks A, B, C, and D of a co-located block. Furthermore,examples of such a new candidate further include a block having a motionvector, a reference picture index, a prediction direction, and the likewhich are statistically obtained for the whole or a certain region of areference picture. Thus, when the total number of merging blockcandidates is smaller than the total number of usable-for-mergingcandidates, the merging block candidate calculation unit 514 adds a newcandidate having a new motion vector, a new reference picture index, anda new prediction direction so that coding efficiency can be increased.

FIG. 28 is a flowchart showing details of the process in Step S511 inFIG. 27. Specifically, FIG. 28 illustrates a method of determiningwhether or not a merging block candidate [N] is a usable-for-mergingcandidate and updating the total number of usable-for-mergingcandidates. FIG. 28 will be described below.

In Step S521, the merging block candidate calculation unit 514determines whether it is true or false that (1) a merging blockcandidate [N] has been coded by intra prediction, (2) the merging blockcandidate [N] is a block outside the boundary of a slice including thecurrent block or the boundary of a picture including the current block,or (3) the merging block candidate [N] is yet to be coded.

When the result of the determination in Step 521 is true (S521, Yes),the merging block candidate calculation unit 514 sets the merging blockcandidate [N] as an unusable-for-merging candidate in Step S522. On theother hand, when the result of the determination in Step S521 is false(S521, No), the merging block candidate calculation unit 514 sets themerging block candidate [N] as a usable-for-merging candidate in StepS523.

In Step S524, the merging block candidate calculation unit 514determines whether it is true or false that the merging block candidate[N] is either a usable-for-merging candidate or a co-located mergingblock candidate. Here, when the result of the determination in Step S524is true (S524, Yes), the merging block candidate calculation unit 514updates the total number of merging block candidates by incrementing itby one in Step S525. On the other hand, when the result of thedetermination in Step S524 is false (S524, No), the merging blockcandidate calculation unit 514 does not update the total number ofusable-for-merging candidates.

Thus, when a merging block candidate is a co-located merging block, themerging block candidate calculation unit 514 increments the total numberof usable-for-merging candidate by one regardless of whether theco-located block is a usable-for-merging candidate or anunusable-for-merging candidate. This prevents discrepancy of the numbersof usable-for-merging candidates between the image coding apparatus andthe image decoding apparatus even when information on a co-locatedmerging block is lost due to an incident such as packet loss.

The total number of usable-for-merging candidates is set as the size ofthe merging block candidate list in Step S515 shown in FIG. 27.Furthermore, the size of the merging block candidate list is used invariable-length coding of merging block candidate indexes in Step S505shown in FIG. 25. This makes it possible for the image coding apparatus500 to generate a bitstream which can be normally decoded so thatmerging block candidate indexes can be obtained even when information onreference picture including a co-located block is lost.

FIG. 29 is a flowchart showing details of the process in Step S514 inFIG. 27. Specifically, FIG. 29 illustrates a method of adding a newcandidate. FIG. 29 will be described below.

In Step S531, the merging block candidate calculation unit 514determines whether or not the total number of merging block candidatesis smaller than the total number of usable-for-merging candidates. Inother words, the merging block candidate calculation unit 514 determineswhether or not the total number of merging block candidate is stillbelow the total number of usable-for-merging candidates.

Here, when the result of the determination in Step S531 is true (S531,Yes), in Step S532, the merging block candidate calculation unit 514determines whether or not there is a new candidate which can be added asa merging block candidate to the merging block candidate list. Here,when the result of the determination in Step S532 is true (S532, Yes),in Step S533, the merging block candidate calculation unit 514 assigns amerging block candidate index to the new candidate and adds the newcandidate to the merging block candidate list. Furthermore, the mergingblock candidate calculation unit 514 increments the total number ofmerging block candidate by one in Step S534.

On the other hand, when the result of the determination in Step S101 orin Step S532 is false (S531 or S532, No), the process for adding a newcandidate ends. In other words, the process for adding a new candidateends when the total number of merging block candidates numbers the totalnumber of usable-for-merging candidates or when there is no newcandidate.

Thus, the image coding apparatus 500 according to Embodiment 5 iscapable of calculating the size of a merging block candidate list foruse in coding or decoding of a merging block candidate index, using amethod independent of information on reference pictures including aco-located block. The image coding apparatus 500 thereby achievesenhanced error resistance.

More specifically, regardless of whether or not a co-located mergingblock is a usable-for-merging candidate, the image coding apparatus 500according to Embodiment 5 increments the total number ofusable-for-merging candidates by one each time a merging block candidateis determined as a co-located merging block. Then, the image codingapparatus 500 determines a bit sequence to be assigned to a mergingblock candidate index, using the total number of usable-for-mergingcandidates calculated in this manner. The image coding apparatus 500 isthus capable of generating a bitstream from which the merging blockcandidate index can be decoded normally even when information onreference pictures including a co-located block is lost.

Furthermore, when the total number of merging block candidates issmaller than the total number of usable-for-merging candidates, theimage coding apparatus 500 according to Embodiment 5 adds, as a mergingblock candidate, a new candidate having a new motion vector, a newreference picture index, and a new prediction direction so that codingefficiency can be increased.

It should be noted that the example described in Embodiment 5 in whichmerging flag is always attached to a bitstream in merging mode is notlimiting. For example, the merging mode may be forcibly selecteddepending on a block shape for use in inter prediction of a currentblock. In this case, it is possible to reduce the amount of informationby attaching no merging flag to a bitstream.

It should be noted that the example described in Embodiment 5 where themerging mode is used in which a current block is coded using aprediction direction, a motion vector, and a reference picture indexcopied from a neighboring block of the current block is not limiting.For example, a skip merging mode may be used. In the skip merging mode,a current block is coded in the same manner as in the merging mode,using a prediction direction, a motion vector, and a reference pictureindex copied from a neighboring block of the current block withreference to a merging block candidate list created as shown in (b) inFIG. 26. When all resultant prediction errors are zero for the currentblock, a skip flag set at 1 and the skip flag and a merging blockcandidate index are attached to a bitstream. When any of the resultantprediction errors is non-zero, a skip flag is set at 0 and the skipflag, a merging flag, a merging block candidate index, and theprediction errors are attached to a bitstream.

It should be noted that the example described in Embodiment 5 where themerging mode is used in which a current block is coded using aprediction direction, a motion vector, and a reference picture indexcopied from a neighboring block of the current block is not limiting.For example, a motion vector in the motion vector estimation mode may becoded using a merging block candidate list created as shown in (b) inFIG. 26. Specifically, a difference is calculated by subtracting amotion vector of a merging block candidate indicated by a merging blockcandidate index from a motion vector in the motion vector estimationmode. Furthermore, the calculated difference and the merging blockcandidate index may be attached to a bitstream.

Optionally, a difference may be calculated by scaling a motion vectorMV_Merge of a merging block candidate using a reference picture indexRefldx_ME in the motion estimation mode and a reference picture indexRefldx_Merge of the merging block candidate as represented by Equation2, and subtracting a motion vector scaledMV_Merge of the merging blockcandidate after the scaling from the motion vector in the motionestimation mode. Then, the calculated difference and the merging blockcandidate index may be attached to a bitstream.

Embodiment 6

In Embodiment 5, the image coding apparatus determines a bit sequence tobe assigned to a merging block candidate index using the total number ofusable-for-merging candidates incremented by one each time a mergingblock candidate is determined as a co-located merging block, regardlessof whether or not a co-located merging block is a usable-for-mergingcandidate. Optionally, for example, the image coding apparatus maydetermine a bit sequence to be assigned to a merging block candidateindex using the total number of usable-for-merging candidates calculatedby incrementing by one for each merging block candidate regardless ofwhether or not the merging block candidate is a co-located merging blockin Step S524 in FIG. 28. In other words, the image coding apparatus mayassign a bit sequence to a merging block candidate index using the sizeof a merging block candidate list fixed at a maximum number N of thetotal number of merging block candidates. In other words, the imagecoding apparatus may code merging block candidate indexes using the sizeof a merging block candidate list fixed at a maximum value N of thetotal number of merging block candidates on the assumption that themerging block candidates are all usable-for-merging candidates.

For example, in the case shown in Embodiment 5, when the maximum value Nof the total number of merging block candidates is five (the neighboringblock A, neighboring block B, co-located merging block, neighboringblock C, and neighboring block D), the image coding apparatus may codethe merging block candidate indexes using the size of the merging blockcandidate list fixedly set at five. Furthermore, for example, when themaximum value N of the total number of merging block candidates is four(the neighboring block A, neighboring block B, neighboring block C, andneighboring block D), the image coding apparatus may code the mergingblock candidate indexes using the size of the merging block candidatelist fixedly set at four.

In this manner, the image coding apparatus may determine the size of amerging block candidate list based on the maximum value of the totalnumber of merging block candidates. It is therefore possible to generatea bitstream from which a variable-length-decoding unit of an imagedecoding apparatus can decode a merging block candidate index withoutreferencing information on a neighboring block or on a co-located block,so that computational complexity for the variable-length-decoding unitcan be reduced.

Such a modification of the image coding apparatus according toEmbodiment 5 will be specifically described below as an image codingapparatus according to Embodiment 6.

FIG. 30 is a block diagram showing a configuration of an image codingapparatus 600 according to Embodiment 6. The image coding apparatus 600codes an image on a block-by-block basis to generate a bitstream. Theimage coding apparatus 600 includes a merging candidate derivation unit610, a prediction control unit 620, and a coding unit 630.

The merging candidate derivation unit 610 corresponds to the mergingblock candidate calculation unit 514 in Embodiment 5. The mergingcandidate derivation unit 610 derives merging candidates. The mergingcandidate derivation unit 610 generates a merging candidate list inwhich, for example, indexes each identifying a different derived mergingcandidate are associated with the respective derived merging candidates.

As shown in FIG. 30, the merging candidate derivation unit 610 includesa first determination unit 611, a first derivation unit 612, aspecification unit 613, a second determination unit 614, and a secondderivation unit 615.

The first determination unit 611 determines a maximum number of mergingcandidates. In other words, the first determination unit 611 determinesa maximum value N of the total number of merging block candidates.

For example, the first determination unit 611 determines a maximumnumber of the merging candidates based on characteristics of the inputimage sequence (such as a sequence, a picture, a slice, or a block).Optionally, for example, the first determination unit 611 may determinea predetermined number as a maximum number of merging candidates.

More specifically, the first derivation unit 612 derives first mergingcandidates based on, for example, a prediction direction, a motionvector, and a reference picture index which have been used for coding ablock spatially or temporally neighboring the current block.Specifically, the first derivation unit 612 derives first mergingcandidates within a range in which the total number of the first mergingcandidates does not exceed the maximum number. Then, for example, thefirst derivation unit 612 registers the first merging candidates derivedin this manner in the merging candidate list in association with therespective merging candidate indexes.

It should be noted that the first derivation unit 612 may derive, as afirst merging candidate, a combination of a prediction direction, amotion vector, and a reference picture index which have been used forcoding blocks which spatially neighbor the current block exceptunusable-for-merging blocks. An unusable-for-merging block is a blockcoded by intra prediction, a block outside the boundary of a sliceincluding the current block or the boundary of a picture including thecurrent block, or a block yet to be coded. With this configuration, thefirst derivation unit 612 can derive first merging candidates fromblocks appropriate for obtaining merging candidates.

When a plurality of first merging candidates has been derived, thespecification unit 613 specifies an identical candidate, that is, afirst merging candidate which is a combination of a predictiondirection, a motion vector, and a reference picture index identical to acombination of a prediction direction, a motion vector, and a referencepicture index of any other of the first merging candidates. Then, thespecification unit 613 removes the specified identical candidate fromthe merging candidate list.

The second determination unit 614 determines whether or not the totalnumber of the first merging candidates is smaller than a determinedmaximum number. Here, the second determination unit 614 determineswhether or not the total number of the first merging candidates exceptthe specified identical first merging candidate is smaller than thedetermined maximum number.

When it is determined that the total number of the first mergingcandidates is smaller than the determined maximum number, the secondderivation unit 615 derives a second merging candidate having a motionvector which is a predetermined vector. Specifically, the secondderivation unit 615 derives second merging candidates within a range inwhich the sum of the total number of first merging candidates and thetotal number of the second merging candidates does not exceed themaximum number. Here, the second derivation unit 615 derives secondmerging candidates within a range in which the sum of the total numberof first merging candidates except the identical candidate and the totalnumber of the second merging candidates does not exceed the maximumnumber.

The predetermined vector may be a zero vector as described in Embodiment5. It should be noted that the predetermined vector need not be a zerovector.

Then, for example, the second derivation unit 615 registers secondmerging candidates derived in this manner in the merging candidate listeach in association with a different merging candidate index. At thistime, the second derivation unit 615 may register the second mergingcandidates in the merging candidate list so that the merging candidateindexes assigned to the first merging candidates are smaller than themerging candidate indexes assigned to the second merging candidates.With this, the image coding apparatus 600 can reduce the code amountwhen the first merging candidates are more likely to be selected as amerging candidate to be used for coding than a second merging candidateso that coding efficiency can be increased.

It should be noted that the second derivation unit 615 need not derive asecond merging candidate so that the sum of the total number of thefirst merging candidates and the total number of the second mergingcandidate equals a determined maximum number. When the sum of the totalnumber of the first merging candidates and the total number of thesecond merging candidate is smaller than the determined maximum number,for example, there may be a merging candidate index with which nomerging candidate is associated.

The prediction control unit 620 selects a merging candidate to be usedfor coding a current block from the first merging candidates and thesecond merging candidates. In other words, the prediction control unit620 selects a merging candidate to be used for coding a current blockfrom the merging candidate list.

The coding unit 630 codes the index for identifying the selected mergingcandidate (merging candidate index) using the determined maximum number.Specifically, the coding unit 630 performs variable-length coding on abit sequence assigned to the index value of the selected mergingcandidate as shown in FIG. 5. Furthermore, the coding unit 630 attachesthe coded index to a bitstream.

Here, the coding unit 630 may further attach information indicating themaximum number determined by the first determination unit 611 to thebitstream. Specifically, for example, the coding unit 630 may write theinformation indicating the maximum number in a slice header. This makesit possible to change maximum numbers by the appropriate unit so thatcoding efficiency can be increased.

The coding unit 630 need not attach a maximum number to a bitstream. Forexample, when the maximum number is specified in a standard, or when themaximum number is the same as a default value, the coding unit 630 neednot attach information indicating the maximum number to a bitstream.

Next, operations of the image coding apparatus 600 in theabove-described configuration will be described below.

FIG. 31 is a flowchart showing processing operations of the image codingapparatus 600 according to Embodiment 6.

First, the first determination unit 611 determines a maximum number ofmerging candidates (S601). The first derivation unit 612 derives a firstmerging candidate (S602). When a plurality of first merging candidateshas been derived, the specification unit 613 specifies a first mergingcandidate which is an identical candidate, that is, a combination of aprediction direction, a motion vector, and a reference picture indexidentical to a combination of a prediction direction, a motion vector,and a reference picture index of any other of the first mergingcandidates (S603).

The second determination unit 614 determines whether or not the totalnumber of the first merging candidates except the identical candidate issmaller than the determined maximum number (S604). Here, when it isdetermined that the total number of the first merging candidates exceptthe identical candidate is smaller than the determined maximum number(S604, Yes), the second derivation unit 615 derives, as a second mergingcandidate, a merging candidate having a motion vector which is apredetermined vector (S605). On the other hand, when it is determinedthat the total number of the first merging candidates except theidentical candidate is not smaller than the determined maximum number(S604, No), the second derivation unit 415 derives no second mergingcandidate. These Step S604 and Step S605 correspond to Step S514 inEmbodiment 5.

The prediction control unit 620 selects a merging candidate to be usedfor coding a current block from the first merging candidates and thesecond merging candidates (S606). For example, the prediction controlunit 620 selects a merging candidate for which the cost represented byEquation 1 is a minimum from the merging candidate list as in Embodiment1.

The coding unit 630 codes an index for identifying the selected mergingcandidate, using the determined maximum number (S607). Furthermore, thecoding unit 630 attaches the coded index to a bitstream.

In this manner, the image coding apparatus 600 according to Embodiment 6can derive, as a second merging candidate, a merging candidate having amotion vector which is a predetermined vector. It is therefore possiblefor the image coding apparatus 600 to derive a merging candidate havinga motion vector and others for a stationary region as a second mergingcandidate, for example. In other words, it is possible for the imagecoding apparatus 600 to efficiently code a current block having apredetermined motion, so that a coding efficiency can be increased.

Furthermore, the image coding apparatus 600 according to Embodiment 6can code an index for identifying a merging candidate using a determinedmaximum number. In other words, an index can be coded independently ofthe total number of actually derived merging candidates. Therefore, evenwhen information necessary for derivation of a merging candidate (forexample, information on a co-located block) is lost, an index can bestill decoded and error resistance is thereby enhanced. Furthermore, anindex can be decoded independently of the total number of actuallyderived merging candidates. In other words, an index can be decodedwithout waiting for derivation of merging candidates. In other words, abitstream can be generated for which deriving of merging candidates anddecoding of indexes can be performed in parallel.

Furthermore, with the image coding apparatus 600 according to Embodiment6, a second merging candidate can be derived when it is determined thatthe total number of the first merging candidates is smaller than themaximum number. Accordingly, the total number of merging candidates canbe increased within a range not exceeding the maximum number so thatcoding efficiency can be increased.

Furthermore, with the image coding apparatus 600 according to Embodiment6, a second merging candidate can be derived based on the total numberof first merging candidates except identical first merging candidates.As a result, the total number of the second merging candidates can beincreased so that the variety of combinations of a prediction direction,a motion vector, and a reference picture index for a selectable mergingcandidate can be increased. It is therefore possible to further increasecoding efficiency.

In Embodiment 6, the specification unit 613 included in the image codingapparatus 600 is not always necessary to the image coding apparatus 600.In other words, Step S603 in the flowchart shown in FIG. 31 is notalways necessary. Even in such a case, the image coding apparatus 600can code an index for identifying a merging candidate using a determinedmaximum number so that error resistance can be enhanced.

Furthermore, in Embodiment 6, although the specification unit 613specifies an identical candidate after the first derivation unit 612derives first merging candidates as shown in FIG. 31, the process neednot be performed in this order. For example, the first derivation unit612 may identify an identical candidate in the process for derivingfirst merging candidates, and derives the first merging candidates suchthat the specified identical candidate is excluded from the firstmerging candidates. In other words, the first derivation unit 612 mayderive, as a first merging candidate, a merging candidate which is acombination of a prediction direction, a motion vector, and a referencepicture index different from a combination of a prediction direction, amotion vector, and a reference picture index of any first mergingcandidate previously derived. More specifically, for example, in thecase where a merging candidate based on a left neighboring block hasalready been selected as a first merging candidate, the first derivationunit 612 may derive a merging candidate which is based on an upperneighboring block as a first merging candidate when the mergingcandidate based on the upper neighboring block is different from themerging candidate which is based on the left neighboring block. In thismanner, the first derivation unit 612 can remove, from the first mergingcandidates, a merging candidate which is a combination of a predictiondirection, a motion vector, and a reference picture index identical to acombination of a prediction direction, a motion vector, and a referencepicture index of any first merging candidate previously derived. As aresult, the image coding apparatus 600 can increase the total number ofthe second merging candidates, and thereby increase the variety ofcombinations of a prediction direction, a motion vector, and a referencepicture index from which a merging candidate is selected. The firstderivation unit 612 thus can further increase coding efficiency.

In Embodiment 6, although deriving of a first merging candidate isfollowed by the determining whether or not the total number of the firstmerging candidate is smaller than a maximum number, and then by derivingof a second merging candidate, the process is not necessarily performedin this order. For example, first, the image coding apparatus 600 mayderive a second merging candidate, and then register the derived secondmerging candidate in a merging candidate list. The image codingapparatus 600 may subsequently derive a first merging candidate, andoverwrite the second merging candidate registered in the mergingcandidate list with the derived first merging candidate.

Embodiment 7

Embodiment 7 is different in the method of deriving the size of amerging block candidate list from Embodiment 3. The method of derivingthe size of a merging block candidate list according to Embodiment 7will be described below in detail.

FIG. 32 is a block diagram showing a configuration of an image decodingapparatus 700 according to Embodiment 7. For FIG. 32, the constituentelements in common with FIG. 20 are denoted with the same referencesigns, and description thereof is omitted.

The image decoding apparatus 700 is an apparatus corresponding to theimage coding apparatus 500 according to Embodiment 5. Specifically, forexample, the image decoding apparatus 700 decodes, on a block-by-blockbasis, coded images included in a bitstream generated by the imagecoding apparatus 500 according to Embodiment 5.

As shown in FIG. 32, the image decoding apparatus 700 includes avariable-length-decoding unit 701, an inverse-quantization unit 302, aninverse-orthogonal-transformation unit 303, an adder 304, block memory305, frame memory 306, an intra prediction unit 307, an inter predictionunit 308, an inter prediction control unit 309, a switch 310, a mergingblock candidate calculation unit 711, and colPic memory 312.

The variable-length-decoding unit 701 generates picture-typeinformation, a merging flag, and a quantized coefficient by performingvariable-length decoding on an input bitstream. Furthermore, thevariable-length-decoding unit 701 obtains a merging block candidateindex by performing variable-length decoding using the total number ofusable-for-merging candidates described below.

The merging block candidate calculation unit 711 derives merging blockcandidates for the merging mode from motion vectors and others ofneighboring blocks of the current block and a motion vector and othersof a co-located block (colPic information) stored in the colPic memory312, using a method described later. Furthermore, the merging blockcandidate calculation unit 711 assigns merging block candidate indexeseach having a different value to the derived merging block candidates.Then, the merging block candidate calculation unit 711 transmits themerging block candidates and merging block candidate indexes to theinter prediction control unit 309.

FIG. 33 is a flowchart showing processing operations of the imagedecoding apparatus according to Embodiment 7.

In Step S701, the variable-length-decoding unit 701 decodes a mergingflag.

In Step S702, when the merging flag is “1” (S702, Yes), in Step S703,the merging block candidate calculation unit 711 calculates the totalnumber of usable-for-merging candidates using a method described later.Then, the merging block candidate calculation unit 711 sets thecalculated number of usable-for-merging candidates as the size of amerging block candidate list.

Next, in Step S704, the variable-length-decoding unit 701 performsvariable-length decoding on a merging block candidate index from abitstream using the size of the merging block candidate list. In StepS705, the merging block candidate calculation unit 711 generates mergingblock candidates from neighboring blocks and a co-located block of acurrent block to be decoded using the method described in Embodiment 1or Embodiment 3 or a method described later.

In Step S706, the inter prediction control unit 309 causes the interprediction unit 308 to generate an inter prediction picture using themotion vector, reference picture index, and prediction direction of themerging block candidate indicated by the decoded merging block candidateindex.

When the merging flag is “0” in Step S702 (Step S702, No), in Step S707,the inter prediction unit 308 generates an inter prediction pictureusing information on motion vector estimation mode decoded by thevariable-length-decoding unit 701.

Optionally, when the size of a merging block candidate list calculatedin Step S703 is “1”, a merging block candidate index may be estimated tobe “0” without being decoded.

FIG. 34 is a flowchart showing details of the process in Step S703 shownin FIG. 33. Specifically, FIG. 34 illustrates a method of determiningwhether or not a merging block candidate [N] is a usable-for-mergingcandidate and calculating the total number of usable-for-mergingcandidates. FIG. 34 will be described below.

In Step S711, the merging block candidate calculation unit 711determines whether it is true or false that (1) a merging blockcandidate [N] has been decoded by intra prediction, (2) the mergingblock candidate [N] is a block outside the boundary of a slice includingthe current block or the boundary of a picture including the currentblock, or (3) the merging block candidate [N] is yet to be decoded.

When the result of the determination in Step S711 is true (S711, Yes),the merging block candidate calculation unit 711 sets the merging blockcandidate [N] as an unusable-for-merging candidate in Step S712. On theother hand, when the result of the determination in Step S711 is false(S711, No), the merging block candidate calculation unit 711 sets themerging block candidate [N] as a usable-for-merging candidate in StepS713.

In Step S714, the merging block candidate calculation unit 711determines whether it is true or false that the merging block candidate[N] is either a usable-for-merging candidate or a co-located mergingblock candidate. Here, when the result of the determination in Step S714is true (S714, Yes), the merging block candidate calculation unit 711updates the total number of merging block candidates by incrementing itby one in Step S715. On the other hand, when the result of thedetermination in Step S714 is false (S714, No), the merging blockcandidate calculation unit 711 does not update the total number ofusable-for-merging candidates.

Thus, when a merging block candidate is a co-located merging block, themerging block candidate calculation unit 711 increments the total numberof usable-for-merging candidates by one regardless of whether theco-located block is a usable-for-merging candidate or anunusable-for-merging candidate. This prevents discrepancy of the numbersof usable-for-merging candidates between the image coding apparatus andthe image decoding apparatus even when information on a co-locatedmerging block is lost due to an incident such as packet loss.

The total number of usable-for-merging candidates is set as the size ofa merging block candidate list in Step S703 shown in FIG. 33.Furthermore, the size of the merging block candidate list is used invariable-length decoding of merging block candidate indexes in Step S704shown in FIG. 33. This makes it possible for the image decodingapparatus 700 to decode merging block candidate indexes normally evenwhen information on reference picture including a co-located block islost.

FIG. 35 is a flowchart showing details of the process in Step S705 shownin FIG. 33. Specifically, FIG. 35 illustrates a method of calculating amerging block candidate. FIG. 35 will be described below.

In Step S721, the merging block candidate calculation unit 711 obtainsthe motion vector, reference picture index, and prediction direction ofa merging block candidate [N], and adds them to a merging blockcandidate list.

In Step S722, the merging block candidate calculation unit 711 searchesthe merging block candidate list for an unusable-for-merging candidateand an identical candidate, and removes the unusable-for-mergingcandidate and the identical candidate from the merging block candidatelist as shown in FIG. 26.

In Step S723, the merging block candidate calculation unit 711 adds anew candidate to the merging block candidate list using the methoddescribed in Embodiment 1 or Embodiment 3 or the method as illustratedin FIG. 29.

FIG. 36 shows exemplary syntax for attachment of merging block candidateindexes to a bitstream. In FIG. 36, merge_idx represents a merging blockcandidate index, and merge_flag represents a merging flag. NumMergeCandrepresents the size of a merging block candidate list. In Embodiment 7,NumMergeCand is set at the total number of usable-for-merging candidatescalculated in the process flow shown in FIG. 34.

Thus, the image decoding apparatus 700 according to Embodiment 7 iscapable of calculating the size of a merging block candidate list foruse in coding or decoding of a merging block candidate index, using amethod independent of information on reference pictures including aco-located block. The image decoding apparatus 700 therefore canappropriately decode a bitstream having enhanced error resistance.

More specifically, regardless of whether or not a co-located mergingblock is a usable-for-merging candidate, the image decoding apparatus700 according to Embodiment 7 increments the total number ofusable-for-merging candidates by one each time a merging block candidateis determined as a co-located merging block. Then, the image decodingapparatus 700 determines a bit sequence assigned to a merging blockcandidate index using the total number of usable-for-merging candidatescalculated in this manner. The image decoding apparatus 700 thus candecode merging block candidate indexes normally even when information onreference picture including a co-located block is lost.

Furthermore, when the total number of the merging block candidates issmaller than the total number of the usable-for-merging candidates, itis possible for the image decoding apparatus 700 according to Embodiment7 to appropriately decode a bitstream coded with increased codingefficiency by adding a new candidate having a new motion vector, a newreference picture index, and a new prediction direction.

Eighth Embodiment

In Embodiment 7, the image decoding apparatus determines a bit sequenceto be assigned to a merging block candidate index using the total numberof usable-for-merging candidates incremented by one each time a mergingblock candidate is determined as a co-located merging block, regardlessof whether or not a co-located merging block is a usable-for-mergingcandidate. Optionally, for example, the image decoding apparatus maydetermine a bit sequence to be assigned to a merging block candidateindex using the total number of usable-for-merging candidates calculatedby incrementing by one for each merging block candidate each mergingblock candidate regardless of whether or not the merging block candidateis a co-located merging block in Step S714 in FIG. 34. In other words,the image decoding apparatus may assign a bit sequence to a mergingblock candidate index using the size of a merging block candidate listfixed at a maximum number N of the total number of merging blockcandidates. In other words, the image decoding apparatus may decodemerging block candidate indexes using the size of a merging blockcandidate list fixed at a maximum value N of the total number of mergingblock candidates on the assumption that the merging block candidates areall usable-for-merging candidates.

For example, in the case shown in Embodiment 7, when the maximum value Nof the total number of merging block candidates is five (the neighboringblock A, neighboring block B, co-located merging block, neighboringblock C, and neighboring block D), the image decoding apparatus maydecode the merging block candidate indexes using the size of the mergingblock candidate list fixedly set at five. It is therefore possible forthe variable-length-decoding unit of the image decoding apparatus todecode a merging block candidate index from a bitstream withoutreferencing information on a neighboring block or on a co-located block.As a result, for example, Step S714 and Step S715 shown in FIG. 34 canbe skipped so that the computational complexity for thevariable-length-decoding unit can be reduced.

FIG. 37 shows exemplary syntax in the case where the size of a mergingblock candidate list is fixed at the maximum value of the total numberof merging block candidates. As can be seen in FIG. 37, NumMergeCand canbe omitted from the syntax when the size of a merging block candidatelist is fixed at a maximum value of the total number of merging blockcandidates.

Such a modification of the image decoding apparatus according toEmbodiment 8 will be specifically described below as an image decodingapparatus according to Embodiment 7.

FIG. 38 is a block diagram showing a configuration of an image decodingapparatus 800 according to Embodiment 8. The image decoding apparatus800 decodes a coded image included in a bitstream on a block-by-blockbasis. Specifically, for example, the image decoding apparatus 800decodes, on a block-by-block basis, coded images included in a bitstreamgenerated by the image coding apparatus 600 according to Embodiment 6.The image decoding apparatus 800 includes a merging candidate derivationunit 810, a decoding unit 820, and a prediction control unit 830.

The merging candidate derivation unit 810 corresponds to the mergingblock candidate calculation unit 711 in Embodiment 7. The mergingcandidate derivation unit 810 derives merging candidates. The mergingcandidate derivation unit 810 generates a merging candidate list inwhich, for example, indexes each identifying a different derived mergingcandidate (merging candidate indexes) are associated with the respectivederived merging candidates.

As shown in FIG. 38, the merging candidate derivation unit 810 includesa first determination unit 811, a first derivation unit 812, aspecification unit 813, a second determination unit 814, and a secondderivation unit 815.

The first determination unit 811 determines a maximum number of mergingcandidates. In other words, the first determination unit 811 determinesa maximum value N of the total number of merging block candidates.

For example, the first determination unit 811 may determine a maximumnumber of the merging candidates using the same method used by the firstdetermination unit 611 in Embodiment 6. Optionally, for example, thefirst determination unit 811 may determine a maximum number based oninformation attached to a bitstream and indicating a maximum number. Theimage decoding apparatus 800 thus can decode an image coded usingmaximum numbers changed by the appropriate unit.

Here, although the first determination unit 811 is included in themerging candidate derivation unit 810, the first determination unit 811may be included in the decoding unit 820.

The first derivation unit 812 derives first merging candidates in thesame manner as the first derivation unit 612 in Embodiment 6.Specifically, the first derivation unit 812 derives first mergingcandidates based on, for example, a prediction direction, a motionvector, and a reference picture index which have been used for decodinga block spatially or temporally neighboring a current block to bedecoded. Then, for example, the first derivation unit 812 registers thefirst merging candidates derived in this manner in the merging candidatelist in association with the respective merging candidate indexes.

It should be noted that the first derivation unit 812 may derive, as afirst merging candidate, a combination of a prediction direction, amotion vector, and a reference picture index which have been used fordecoding blocks which spatially neighbor the current block exceptunusable-for-merging blocks. With this configuration, the firstderivation unit 812 can derive first merging candidates from blocksappropriate for obtaining merging candidates.

When a plurality of first merging candidates has been derived, thespecification unit 813 specifies an identical candidate, that is, afirst merging candidate which is a combination of a predictiondirection, a motion vector, and a reference picture index identical to acombination of a prediction direction, a motion vector, and a referencepicture index of any other of the first merging candidates. Then, thespecification unit 813 removes the specified identical candidate fromthe merging candidate list.

The second determination unit 814 determines whether or not the totalnumber of the first merging candidates is smaller than a determinedmaximum number. Here, the second determination unit 814 determineswhether or not the total number of the first merging candidates exceptthe specified identical first merging candidate is smaller than thedetermined maximum number.

When it is determined that the total number of the first mergingcandidates is smaller than the determined maximum number, the secondderivation unit 815 derives a second merging candidate having a motionvector which is a predetermined vector. Specifically, the secondderivation unit 815 derives second merging candidates within a range inwhich the sum of the total number of first merging candidates and thetotal number of the second merging candidates does not exceed themaximum number. Here, the second derivation unit 815 derives secondmerging candidates within a range in which the sum of the total numberof first merging candidates except the identical candidate and the totalnumber of the second merging candidates does not exceed the maximumnumber.

The predetermined vector may be a zero vector as described in Embodiment7. In this manner, the second derivation unit 815 thus can derive amerging candidate having a motion vector for a stationary region. Theimage decoding apparatus 800 thus can appropriately decode a bitstreamcoded with increased coding efficiency. It should be noted that thepredetermined vector need not be a zero vector.

Then, for example, the second derivation unit 815 registers secondmerging candidates derived in this manner in the merging candidate listeach in association with a different merging candidate index. At thistime, the second derivation unit 815 may register the second mergingcandidates in the merging candidate list so that the merging candidateindexes assigned to the first merging candidates are smaller than themerging candidate indexes assigned to the second merging candidates.With this, the image decoding apparatus 800 can appropriately decode abitstream coded with increased coding efficiency.

It should be noted that the second derivation unit 815 need not derive asecond merging candidate so that the sum of the total number of thefirst merging candidates and the total number of the second mergingcandidate equals a determined maximum number. When the sum of the totalnumber of the first merging candidates and the total number of thesecond merging candidate is smaller than the determined maximum number,for example, there may be a merging candidate index with which nomerging candidate is associated.

The decoding unit 820 decodes an index coded and attached to abitstream, which is an index for identifying a merging candidate, usingthe determined maximum number.

The prediction control unit 830 selects, based on the decoded index, amerging candidate to be used for decoding a current block from the firstmerging candidates and second merging candidates. In other words, theprediction control unit 830 selects a merging candidate to be used fordecoding a current block from the merging candidate list.

Next, operations of the image decoding apparatus 800 in theabove-described configuration will be explained below.

FIG. 39 is a flowchart showing processing operations of the imagedecoding apparatus 800 according to Embodiment 8.

First, the first determination unit 811 determines a maximum number ofmerging candidates (S801). The first derivation unit 812 derives a firstmerging candidate (S802). When a plurality of first merging candidateshas been derived, the specification unit 813 specifies a first mergingcandidate which is an identical candidate, that is, a combination of aprediction direction, a motion vector, and a reference picture indexidentical to a combination of a prediction direction, a motion vector,and a reference picture index of any other of the first mergingcandidates (S803).

The second determination unit 814 determines whether or not the totalnumber of the first merging candidates except the identical candidate issmaller than the determined maximum number (S804). Here, when it isdetermined that the total number of the first merging candidates exceptthe identical candidate is smaller than the determined maximum number(S804, Yes), the second derivation unit 815 derives second mergingcandidates (S805). On the other hand, when it is determined that thetotal number of the first merging candidates except the identicalcandidate is not smaller than the determined maximum number (S804, No),the second derivation unit 815 derives no second merging candidate.

The decoding unit 820 decodes an index coded and attached to abitstream, which is an index for identifying a merging candidate, usingthe determined maximum number (S806).

The prediction control unit 830 selects, based on the decoded index, amerging candidate to be used for decoding a current block from the firstmerging candidates and second merging candidates (S807). For example,the prediction control unit 830 selects a merging candidate for whichthe cost represented by Equation 1 is a minimum from the mergingcandidate list as in Embodiment 1.

Although the process is performed such that the decoding an index (S806)is performed after a merging candidate is derived, the process need notbe performed in this order. For example, a merging candidate may bederived (S802 to S805) after decoding an index (S806). Optionally,decoding an index (S806) and deriving of a merging candidate (S802 toS805) may be performed in parallel. This increases processing speed fordecoding.

In this manner, the image decoding apparatus 800 according to Embodiment8 can derive, as a second merging candidate, a merging candidate havinga motion vector which is a predetermined vector. It is thereforepossible for the image decoding apparatus 800 to derive a mergingcandidate having a motion vector and others for a stationary region as asecond merging candidate, for example. In other words, it is possiblefor the image decoding apparatus 800 to efficiently decode a currentblock having a predetermined motion, and thereby appropriately decode abitstream coded with increased coding efficiency.

Furthermore, the image decoding apparatus 800 according to Embodiment 8can decode an index for identifying a merging candidate, using adetermined maximum number. In other words, an index can be decodedindependently of the total number of actually derived mergingcandidates. Therefore, even when information necessary for derivation ofa merging candidate (for example, information on a co-located block) islost, the image decoding apparatus 800 still can decode an index, anderror resistance is thereby enhanced. Furthermore, the image decodingapparatus 800 can decode an index without waiting for derivation ofmerging candidates so that deriving of merging candidates and decodingof indexes can be performed in parallel.

Furthermore, the image decoding apparatus 800 according to Embodiment 8can derive a second merging candidate when it is determined that thetotal number of the first merging candidates is smaller than a maximumnumber. Accordingly, the image decoding apparatus 800 can increase thetotal number of merging candidates within a range not exceeding themaximum number, and appropriately decode a bitstream coded withincreased coding efficiency.

Furthermore, the image decoding apparatus 800 according to Embodiment 8can derive a second merging candidate based on the total number of firstmerging candidates except identical first merging candidates. As aresult, the image decoding apparatus 800 can increase the total numberof the second merging candidates, and thereby increase the variety ofcombinations of a prediction direction, a motion vector, and a referencepicture index from which a merging candidate is selected. With this, theimage decoding apparatus 800 thus can appropriately decode a bitstreamcoded with further increased coding efficiency.

As in Embodiment 6, the specification unit 813 included in the imagedecoding apparatus 800 is not always necessary to the image decodingapparatus 800 in Embodiment 8. In other words, Step S803 in theflowchart shown in FIG. 39 is not always necessary. Even in such a case,the image decoding apparatus 800 can decode an index for identifying amerging candidate using a determined maximum number so that errorresistance can be enhanced.

Furthermore, in Embodiment 8, although the specification unit 813specifies an identical candidate after the first derivation unit 812derives first merging candidates as shown in FIG. 39, the process neednot be performed in this order. For example, the first derivation unit812 may derive, as a first merging candidate, a merging candidate whichis a combination of a prediction direction, a motion vector, and areference picture index different from a combination of a predictiondirection, a motion vector, and a reference picture index of any firstmerging candidate previously derived. In this manner, the firstderivation unit 812 can remove, from the first merging candidates, amerging candidate which is a combination of a prediction direction, amotion vector, and a reference picture index identical to a combinationof a prediction direction, a motion vector, and a reference pictureindex of any first merging candidate previously derived. As a result,the image decoding apparatus 800 can increase the total number of thesecond merging candidates, and thereby increase the variety ofcombinations of a prediction direction, a motion vector, and a referencepicture index from which a merging candidate is selected. The imagedecoding apparatus 800 thus can appropriately decode a bitstream codedwith further increased coding efficiency.

In Embodiment 8, although deriving of a first merging candidate isfollowed by the determining whether or not the total number of the firstmerging candidate is smaller than a maximum number, and then by derivingof a second merging candidate, the process is not necessarily performedin this order. For example, first, the image decoding apparatus 800 mayderive a second merging candidate, and then register the derived secondmerging candidate in a merging candidate list. The image decodingapparatus 800 may subsequently derive a first merging candidate, andoverwrite the second merging candidate registered in the mergingcandidate list with the derived first merging candidate.

Although the image coding apparatus and image decoding apparatusaccording to one or more aspects of the present disclosure have beendescribed based on the embodiments, the present disclosure is notlimited to the embodiments. Those skilled in the art will readilyappreciate that many modifications of the exemplary embodiments orembodiments in which the constituent elements of the exemplaryembodiments are combined are possible without materially departing fromthe novel teachings and advantages described in the present disclosure.All such modifications and embodiments are also within scopes of one ormore aspects of the present disclosure.

In the exemplary embodiments, each of the constituent elements may beimplemented as a piece of dedicated hardware or implemented by executinga software program appropriate for the constituent element. Theconstituent elements may be implemented by a program execution unit suchas a CPU or a processor which reads and executes a software programrecorded on a recording medium such as a hard disk or a semiconductormemory. Here, examples of the software program which implements theimage coding apparatus or image decoding apparatus in the embodimentsinclude a program as follows.

Specifically, the program causes a computer to execute an image codingmethod which is a method for coding an image on a block-by-block basisto generate a bitstream and includes: deriving, as a first mergingcandidate, a merging candidate which is a combination of a predictiondirection, a motion vector, and a reference picture index for use incoding of the current block; deriving, as a second merging candidate, amerging candidate having a motion vector which is a predeterminedvector; selecting a merging candidate to be used for the coding of thecurrent block from the derived first merging candidate and the derivedsecond merging candidate; and attaching an index for identifying theselected merging candidate to the bitstream.

Furthermore, the program causes a computer to execute an image decodingmethod which is a method for decoding, on a block-by-block basis, acoded image included in a bitstream and includes: deriving, as a firstmerging candidate, a merging candidate which is a combination of aprediction direction, a motion vector, and a reference picture index foruse in decoding of the current block; deriving, as a second mergingcandidate, a merging candidate having a motion vector which is apredetermined vector; obtaining, from the bitstream, an index foridentifying a merging candidate; and selecting, based on the obtainedindex, a merging candidate from the first merging candidate and thesecond merging candidate, the merging candidate being to be used for thedecoding of a current block.

Embodiment 9

The processing described in each of embodiments can be simplyimplemented in an independent computer system, by recording, in arecording medium, a program for implementing the configurations of themoving picture coding method (image coding method) and the movingpicture decoding method (image decoding method) described in each ofembodiments. The recording media may be any recording media as long asthe program can be recorded, such as a magnetic disk, an optical disk, amagnetic optical disk, an IC card, and a semiconductor memory.

Hereinafter, the applications to the moving picture coding method (imagecoding method) and the moving picture decoding method (image decodingmethod) described in each of embodiments and systems using thereof willbe described. The system has a feature of having an image coding anddecoding apparatus that includes an image coding apparatus using theimage coding method and an image decoding apparatus using the imagedecoding method. Other configurations in the system can be changed asappropriate depending on the cases.

FIG. 40 illustrates an overall configuration of a content providingsystem ex100 for implementing content distribution services. The areafor providing communication services is divided into cells of desiredsize, and base stations ex106, ex107, ex108, ex109, and ex110 which arefixed wireless stations are placed in each of the cells.

The content providing system ex100 is connected to devices, such as acomputer ex111, a personal digital assistant (PDA) ex112, a cameraex113, a cellular phone ex114 and a game machine ex115, via the Internetex101, an Internet service provider ex102, a telephone network ex104, aswell as the base stations ex106 to ex110, respectively.

However, the configuration of the content providing system ex100 is notlimited to the configuration shown in FIG. 40, and a combination inwhich any of the elements are connected is acceptable. In addition, eachdevice may be directly connected to the telephone network ex104, ratherthan via the base stations ex106 to ex110 which are the fixed wirelessstations. Furthermore, the devices may be interconnected to each othervia a short distance wireless communication and others.

The camera ex113, such as a digital video camera, is capable ofcapturing video. A camera ex116, such as a digital camera, is capable ofcapturing both still images and video. Furthermore, the cellular phoneex114 may be the one that meets any of the standards such as GlobalSystem for Mobile Communications (GSM) (registered trademark), CodeDivision Multiple Access (CDMA), Wideband-Code Division Multiple Access(W-CDMA), Long Term Evolution (LTE), and High Speed Packet Access(HSPA). Alternatively, the cellular phone ex114 may be a PersonalHandyphone System (PHS).

In the content providing system ex100, a streaming server ex103 isconnected to the camera ex113 and others via the telephone network ex104and the base station ex109, which enables distribution of images of alive show and others. In such a distribution, a content (for example,video of a music live show) captured by the user using the camera ex113is coded as described above in each of embodiments (i.e., the camerafunctions as the image coding apparatus according to an aspect of thepresent disclosure), and the coded content is transmitted to thestreaming server ex103. On the other hand, the streaming server ex103carries out stream distribution of the transmitted content data to theclients upon their requests. The clients include the computer ex111, thePDA ex112, the camera ex113, the cellular phone ex114, and the gamemachine ex115 that are capable of decoding the above-mentioned codeddata. Each of the devices that have received the distributed datadecodes and reproduces the coded data (i.e., functions as the imagedecoding apparatus according to an aspect of the present disclosure).

The captured data may be coded by the camera ex113 or the streamingserver ex103 that transmits the data, or the coding processes may beshared between the camera ex113 and the streaming server ex103.Similarly, the distributed data may be decoded by the clients or thestreaming server ex103, or the decoding processes may be shared betweenthe clients and the streaming server ex103. Furthermore, the data of thestill images and video captured by not only the camera ex113 but alsothe camera ex116 may be transmitted to the streaming server ex103through the computer ex111. The coding processes may be performed by thecamera ex116, the computer ex111, or the streaming server ex103, orshared among them.

Furthermore, the coding and decoding processes may be performed by anLSI ex500 generally included in each of the computer ex111 and thedevices. The LSI ex500 may be configured of a single chip or a pluralityof chips. Software for coding and decoding video may be integrated intosome type of a recording medium (such as a CD-ROM, a flexible disk, anda hard disk) that is readable by the computer ex111 and others, and thecoding and decoding processes may be performed using the software.Furthermore, when the cellular phone ex114 is equipped with a camera,the video data obtained by the camera may be transmitted. The video datais data coded by the LSI ex500 included in the cellular phone ex114.

Furthermore, the streaming server ex103 may be composed of servers andcomputers, and may decentralize data and process the decentralized data,record, or distribute data.

Furthermore, the streaming server ex103 may be composed of servers andcomputers, and may decentralize data and process the decentralized data,record, or distribute data.

As described above, the clients may receive and reproduce the coded datain the content providing system ex100. In other words, the clients canreceive and decode information transmitted by the user, and reproducethe decoded data in real time in the content providing system ex100, sothat the user who does not have any particular right and equipment canimplement personal broadcasting.

Aside from the example of the content providing system ex100, at leastone of the moving picture coding apparatus (image coding apparatus) andthe moving picture decoding apparatus (image decoding apparatus)described in each of embodiments may be implemented in a digitalbroadcasting system ex200 illustrated in FIG. 41. More specifically, abroadcast station ex201 communicates or transmits, via radio waves to abroadcast satellite ex202, multiplexed data obtained by multiplexingaudio data and others onto video data. The video data is data coded bythe moving picture coding method described in each of embodiments (i.e.,data coded by the image coding apparatus according to an aspect of thepresent disclosure). Upon receipt of the multiplexed data, the broadcastsatellite ex202 transmits radio waves for broadcasting. Then, a home-useantenna ex204 with a satellite broadcast reception function receives theradio waves. Next, a device such as a television (receiver) ex300 and aset top box (STB) ex217 decodes the received multiplexed data, andreproduces the decoded data (i.e., functions as the image decodingapparatus according to an aspect of the present disclosure).

Furthermore, a reader/recorder ex218 (i) reads and decodes themultiplexed data recorded on a recording medium ex215, such as a DVD anda BD, or (i) codes video signals in the recording medium ex215, and insome cases, writes data obtained by multiplexing an audio signal on thecoded data. The reader/recorder ex218 can include the moving picturedecoding apparatus or the moving picture coding apparatus as shown ineach of embodiments. In this case, the reproduced video signals aredisplayed on the monitor ex219, and can be reproduced by another deviceor system using the recording medium ex215 on which the multiplexed datais recorded. It is also possible to implement the moving picturedecoding apparatus in the set top box ex217 connected to the cable ex203for a cable television or to the antenna ex204 for satellite and/orterrestrial broadcasting, so as to display the video signals on themonitor ex219 of the television ex300. The moving picture decodingapparatus may be implemented not in the set top box but in thetelevision ex300.

FIG. 42 illustrates the television (receiver) ex300 that uses the movingpicture coding method and the moving picture decoding method describedin each of embodiments. The television ex300 includes: a tuner ex301that obtains or provides multiplexed data obtained by multiplexing audiodata onto video data, through the antenna ex204 or the cable ex203, etc.that receives a broadcast; a modulation/demodulation unit ex302 thatdemodulates the received multiplexed data or modulates data intomultiplexed data to be supplied outside; and amultiplexing/demultiplexing unit ex303 that demultiplexes the modulatedmultiplexed data into video data and audio data, or multiplexes videodata and audio data coded by a signal processing unit ex306 into data.

The television ex300 further includes: a signal processing unit ex306including an audio signal processing unit ex304 and a video signalprocessing unit ex305 that decode audio data and video data and codeaudio data and video data, respectively (which function as the imagecoding apparatus and the image decoding apparatus according to theaspects of the present disclosure); and an output unit ex309 including aspeaker ex307 that provides the decoded audio signal, and a display unitex308 that displays the decoded video signal, such as a display.Furthermore, the television ex300 includes an interface unit ex317including an operation input unit ex312 that receives an input of a useroperation. Furthermore, the television ex300 includes a control unitex310 that controls overall each constituent element of the televisionex300, and a power supply circuit unit ex311 that supplies power to eachof the elements. Other than the operation input unit ex312, theinterface unit ex317 may include: a bridge ex313 that is connected to anexternal device, such as the reader/recorder ex218; a slot unit ex314for enabling attachment of the recording medium ex216, such as an SDcard; a driver ex315 to be connected to an external recording medium,such as a hard disk; and a modem ex316 to be connected to a telephonenetwork. Here, the recording medium ex216 can electrically recordinformation using a non-volatile/volatile semiconductor memory elementfor storage. The constituent elements of the television ex300 areconnected to each other through a synchronous bus.

First, the configuration in which the television ex300 decodesmultiplexed data obtained from outside through the antenna ex204 andothers and reproduces the decoded data will be described. In thetelevision ex300, upon a user operation through a remote controllerex220 and others, the multiplexing/demultiplexing unit ex303demultiplexes the multiplexed data demodulated by themodulation/demodulation unit ex302, under control of the control unitex310 including a CPU. Furthermore, the audio signal processing unitex304 decodes the demultiplexed audio data, and the video signalprocessing unit ex305 decodes the demultiplexed video data, using thedecoding method described in each of embodiments, in the televisionex300. The output unit ex309 provides the decoded video signal and audiosignal outside, respectively. When the output unit ex309 provides thevideo signal and the audio signal, the signals may be temporarily storedin buffers ex318 and ex319, and others so that the signals arereproduced in synchronization with each other. Furthermore, thetelevision ex300 may read multiplexed data not through a broadcast andothers but from the recording media ex215 and ex216, such as a magneticdisk, an optical disk, and a SD card. Next, a configuration in which thetelevision ex300 codes an audio signal and a video signal, and transmitsthe data outside or writes the data on a recording medium will bedescribed. In the television ex300, upon a user operation through theremote controller ex220 and others, the audio signal processing unitex304 codes an audio signal, and the video signal processing unit ex305codes a video signal, under control of the control unit ex310 using thecoding method described in each of embodiments. Themultiplexing/demultiplexing unit ex303 multiplexes the coded videosignal and audio signal, and provides the resulting signal outside. Whenthe multiplexing/demultiplexing unit ex303 multiplexes the video signaland the audio signal, the signals may be temporarily stored in thebuffers ex320 and ex321, and others so that the signals are reproducedin synchronization with each other. Here, the buffers ex318, ex319,ex320, and ex321 may be plural as illustrated, or at least one buffermay be shared in the television ex300. Furthermore, data may be storedin a buffer so that the system overflow and underflow may be avoidedbetween the modulation/demodulation unit ex302 and themultiplexing/demultiplexing unit ex303, for example.

Furthermore, the television ex300 may include a configuration forreceiving an AV input from a microphone or a camera other than theconfiguration for obtaining audio and video data from a broadcast or arecording medium, and may code the obtained data. Although thetelevision ex300 can code, multiplex, and provide outside data in thedescription, it may be capable of only receiving, decoding, andproviding outside data but not the coding, multiplexing, and providingoutside data.

Furthermore, when the reader/recorder ex218 reads or writes multiplexeddata from or on a recording medium, one of the television ex300 and thereader/recorder ex218 may decode or code the multiplexed data, and thetelevision ex300 and the reader/recorder ex218 may share the decoding orcoding.

As an example, FIG. 43 illustrates a configuration of an informationreproducing/recording unit ex400 when data is read or written from or onan optical disk. The information reproducing/recording unit ex400includes constituent elements ex401, ex402, ex403, ex404, ex405, ex406,and ex407 to be described hereinafter. The optical head ex401 irradiatesa laser spot in a recording surface of the recording medium ex215 thatis an optical disk to write information, and detects reflected lightfrom the recording surface of the recording medium ex215 to read theinformation. The modulation recording unit ex402 electrically drives asemiconductor laser included in the optical head ex401, and modulatesthe laser light according to recorded data. The reproductiondemodulating unit ex403 amplifies a reproduction signal obtained byelectrically detecting the reflected light from the recording surfaceusing a photo detector included in the optical head ex401, anddemodulates the reproduction signal by separating a signal componentrecorded on the recording medium ex215 to reproduce the necessaryinformation. The buffer ex404 temporarily holds the information to berecorded on the recording medium ex215 and the information reproducedfrom the recording medium ex215. The disk motor ex405 rotates therecording medium ex215. The servo control unit ex406 moves the opticalhead ex401 to a predetermined information track while controlling therotation drive of the disk motor ex405 so as to follow the laser spot.The system control unit ex407 controls overall the informationreproducing/recording unit ex400. The reading and writing processes canbe implemented by the system control unit ex407 using variousinformation stored in the buffer ex404 and generating and adding newinformation as necessary, and by the modulation recording unit ex402,the reproduction demodulating unit ex403, and the servo control unitex406 that record and reproduce information through the optical headex401 while being operated in a coordinated manner. The system controlunit ex407 includes, for example, a microprocessor, and executesprocessing by causing a computer to execute a program for read andwrite.

Although the optical head ex401 irradiates a laser spot in thedescription, it may perform high-density recording using near fieldlight.

FIG. 44 illustrates the recording medium ex215 that is the optical disk.On the recording surface of the recording medium ex215, guide groovesare spirally formed, and an information track ex230 records, in advance,address information indicating an absolute position on the diskaccording to change in a shape of the guide grooves. The addressinformation includes information for determining positions of recordingblocks ex231 that are a unit for recording data. Reproducing theinformation track ex230 and reading the address information in anapparatus that records and reproduces data can lead to determination ofthe positions of the recording blocks. Furthermore, the recording mediumex215 includes a data recording area ex233, an inner circumference areaex232, and an outer circumference area ex234. The data recording areaex233 is an area for use in recording the user data. The innercircumference area ex232 and the outer circumference area ex234 that areinside and outside of the data recording area ex233, respectively arefor specific use except for recording the user data. The informationreproducing/recording unit 400 reads and writes coded audio, coded videodata, or multiplexed data obtained by multiplexing the coded audio andvideo data, from and on the data recording area ex233 of the recordingmedium ex215.

Although an optical disk having a layer, such as a DVD and a BD isdescribed as an example in the description, the optical disk is notlimited to such, and may be an optical disk having a multilayerstructure and capable of being recorded on a part other than thesurface. Furthermore, the optical disk may have a structure formultidimensional recording/reproduction, such as recording ofinformation using light of colors with different wavelengths in the sameportion of the optical disk and for recording information havingdifferent layers from various angles.

Furthermore, a car ex210 having an antenna ex205 can receive data fromthe satellite ex202 and others, and reproduce video on a display devicesuch as a car navigation system ex211 set in the car ex210, in thedigital broadcasting system ex200. Here, a configuration of the carnavigation system ex211 will be a configuration, for example, includinga GPS receiving unit from the configuration illustrated in FIG. 42. Thesame will be true for the configuration of the computer ex111, thecellular phone ex114, and others.

FIG. 45A illustrates the cellular phone ex114 that uses the movingpicture coding method and the moving picture decoding method describedin embodiments. The cellular phone ex114 includes: an antenna ex350 fortransmitting and receiving radio waves through the base station ex110; acamera unit ex365 capable of capturing moving and still images; and adisplay unit ex358 such as a liquid crystal display for displaying thedata such as decoded video captured by the camera unit ex365 or receivedby the antenna ex350. The cellular phone ex114 further includes: a mainbody unit including an operation key unit ex366; an audio output unitex357 such as a speaker for output of audio; an audio input unit ex356such as a microphone for input of audio; a memory unit ex367 for storingcaptured video or still pictures, recorded audio, coded or decoded dataof the received video, the still pictures, e-mails, or others; and aslot unit ex364 that is an interface unit for a recording medium thatstores data in the same manner as the memory unit ex367.

Next, an example of a configuration of the cellular phone ex114 will bedescribed with reference to FIG. 45B. In the cellular phone ex114, amain control unit ex360 designed to control overall each unit of themain body including the display unit ex358 as well as the operation keyunit ex366 is connected mutually, via a synchronous bus ex370, to apower supply circuit unit ex361, an operation input control unit ex362,a video signal processing unit ex355, a camera interface unit ex363, aliquid crystal display (LCD) control unit ex359, amodulation/demodulation unit ex352, a multiplexing/demultiplexing unitex353, an audio signal processing unit ex354, the slot unit ex364, andthe memory unit ex367.

When a call-end key or a power key is turned ON by a user's operation,the power supply circuit unit ex361 supplies the respective units withpower from a battery pack so as to activate the cell phone ex114.

In the cellular phone ex114, the audio signal processing unit ex354converts the audio signals collected by the audio input unit ex356 invoice conversation mode into digital audio signals under the control ofthe main control unit ex360 including a CPU, ROM, and RAM. Then, themodulation/demodulation unit ex352 performs spread spectrum processingon the digital audio signals, and the transmitting and receiving unitex351 performs digital-to-analog conversion and frequency conversion onthe data, so as to transmit the resulting data via the antenna ex350.Also, in the cellular phone ex114, the transmitting and receiving unitex351 amplifies the data received by the antenna ex350 in voiceconversation mode and performs frequency conversion and theanalog-to-digital conversion on the data. Then, themodulation/demodulation unit ex352 performs inverse spread spectrumprocessing on the data, and the audio signal processing unit ex354converts it into analog audio signals, so as to output them via theaudio output unit ex357.

Furthermore, when an e-mail in data communication mode is transmitted,text data of the e-mail inputted by operating the operation key unitex366 and others of the main body is sent out to the main control unitex360 via the operation input control unit ex362. The main control unitex360 causes the modulation/demodulation unit ex352 to perform spreadspectrum processing on the text data, and the transmitting and receivingunit ex351 performs the digital-to-analog conversion and the frequencyconversion on the resulting data to transmit the data to the basestation ex110 via the antenna ex350. When an e-mail is received,processing that is approximately inverse to the processing fortransmitting an e-mail is performed on the received data, and theresulting data is provided to the display unit ex358.

When video, still images, or video and audio in data communication modeis or are transmitted, the video signal processing unit ex355 compressesand codes video signals supplied from the camera unit ex365 using themoving picture coding method shown in each of embodiments (i.e.,functions as the image coding apparatus according to the aspect of thepresent disclosure), and transmits the coded video data to themultiplexing/demultiplexing unit ex353. In contrast, during when thecamera unit ex365 captures video, still images, and others, the audiosignal processing unit ex354 codes audio signals collected by the audioinput unit ex356, and transmits the coded audio data to themultiplexing/demultiplexing unit ex353.

The multiplexing/demultiplexing unit ex353 multiplexes the coded videodata supplied from the video signal processing unit ex355 and the codedaudio data supplied from the audio signal processing unit ex354, using apredetermined method. Then, the modulation/demodulation unit(modulation/demodulation circuit unit) ex352 performs spread spectrumprocessing on the multiplexed data, and the transmitting and receivingunit ex351 performs digital-to-analog conversion and frequencyconversion on the data so as to transmit the resulting data via theantenna ex350.

When receiving data of a video file which is linked to a Web page andothers in data communication mode or when receiving an e-mail with videoand/or audio attached, in order to decode the multiplexed data receivedvia the antenna ex350, the multiplexing/demultiplexing unit ex353demultiplexes the multiplexed data into a video data bitstream and anaudio data bitstream, and supplies the video signal processing unitex355 with the coded video data and the audio signal processing unitex354 with the coded audio data, through the synchronous bus ex370. Thevideo signal processing unit ex355 decodes the video signal using amoving picture decoding method corresponding to the moving picturecoding method shown in each of embodiments (i.e., functions as the imagedecoding apparatus according to the aspect of the present disclosure),and then the display unit ex358 displays, for instance, the video andstill images included in the video file linked to the Web page via theLCD control unit ex359. Furthermore, the audio signal processing unitex354 decodes the audio signal, and the audio output unit ex357 providesthe audio.

Furthermore, similarly to the television ex300, a terminal such as thecellular phone ex114 probably have 3 types of implementationconfigurations including not only (i) a transmitting and receivingterminal including both a coding apparatus and a decoding apparatus, butalso (ii) a transmitting terminal including only a coding apparatus and(iii) a receiving terminal including only a decoding apparatus. Althoughthe digital broadcasting system ex200 receives and transmits themultiplexed data obtained by multiplexing audio data onto video data inthe description, the multiplexed data may be data obtained bymultiplexing not audio data but character data related to video ontovideo data, and may be not multiplexed data but video data itself.

As such, the moving picture coding method and the moving picturedecoding method in each of embodiments can be used in any of the devicesand systems described. Thus, the advantages described in each ofembodiments can be obtained.

Furthermore, the present disclosure is not limited to embodiments, andvarious modifications and revisions are possible without departing fromthe scope of the present disclosure.

Embodiment 10

Video data can be generated by switching, as necessary, between (i) themoving picture coding method or the moving picture coding apparatusshown in each of embodiments and (ii) a moving picture coding method ora moving picture coding apparatus in conformity with a differentstandard, such as MPEG-2, MPEG-4 AVC, and VC-1.

Here, when a plurality of video data that conforms to the differentstandards is generated and is then decoded, the decoding methods need tobe selected to conform to the different standards. However, since towhich standard each of the plurality of the video data to be decodedconforms cannot be detected, there is a problem that an appropriatedecoding method cannot be selected.

In order to solve the problem, multiplexed data obtained by multiplexingaudio data and others onto video data has a structure includingidentification information indicating to which standard the video dataconforms. The specific structure of the multiplexed data including thevideo data generated in the moving picture coding method and by themoving picture coding apparatus shown in each of embodiments will behereinafter described. The multiplexed data is a digital stream in theMPEG-2 Transport Stream format.

FIG. 46 illustrates a structure of the multiplexed data. As illustratedin FIG. 46, the multiplexed data can be obtained by multiplexing atleast one of a video stream, an audio stream, a presentation graphicsstream (PG), and an interactive graphics stream. The video streamrepresents primary video and secondary video of a movie, the audiostream (IG) represents a primary audio part and a secondary audio partto be mixed with the primary audio part, and the presentation graphicsstream represents subtitles of the movie. Here, the primary video isnormal video to be displayed on a screen, and the secondary video isvideo to be displayed on a smaller window in the primary video.Furthermore, the interactive graphics stream represents an interactivescreen to be generated by arranging the GUI components on a screen. Thevideo stream is coded in the moving picture coding method or by themoving picture coding apparatus shown in each of embodiments, or in amoving picture coding method or by a moving picture coding apparatus inconformity with a conventional standard, such as MPEG-2, MPEG-4 AVC, andVC-1. The audio stream is coded in accordance with a standard, such asDolby-AC-3, Dolby Digital Plus, MLP, DTS, DTS-HD, and linear PCM.

Each stream included in the multiplexed data is identified by PID. Forexample, 0x1011 is allocated to the video stream to be used for video ofa movie, 0x1100 to 0x111F are allocated to the audio streams, 0x1200 to0x121F are allocated to the presentation graphics streams, 0x1400 to0x141F are allocated to the interactive graphics streams, 0x1B00 to0x1B1F are allocated to the video streams to be used for secondary videoof the movie, and 0x1A00 to 0x1A1F are allocated to the audio streams tobe used for the secondary audio to be mixed with the primary audio.

FIG. 47 schematically illustrates how data is multiplexed. First, avideo stream ex235 composed of video frames and an audio stream ex238composed of audio frames are transformed into a stream of PES packetsex236 and a stream of PES packets ex239, and further into TS packetsex237 and TS packets ex240, respectively. Similarly, data of apresentation graphics stream ex241 and data of an interactive graphicsstream ex244 are transformed into a stream of PES packets ex242 and astream of PES packets ex245, and further into TS packets ex243 and TSpackets ex246, respectively. These TS packets are multiplexed into astream to obtain multiplexed data ex247.

FIG. 48 illustrates how a video stream is stored in a stream of PESpackets in more detail. The first bar in FIG. 48 shows a video framestream in a video stream. The second bar shows the stream of PESpackets. As indicated by arrows denoted as yy1, yy2, yy3, and yy4 inFIG. 48, the video stream is divided into pictures as I pictures, Bpictures, and P pictures each of which is a video presentation unit, andthe pictures are stored in a payload of each of the PES packets. Each ofthe PES packets has a PES header, and the PES header stores aPresentation Time-Stamp (PTS) indicating a display time of the picture,and a Decoding Time-Stamp (DTS) indicating a decoding time of thepicture.

FIG. 49 illustrates a format of TS packets to be finally written on themultiplexed data. Each of the TS packets is a 188-byte fixed lengthpacket including a 4-byte TS header having information, such as a PIDfor identifying a stream and a 184-byte TS payload for storing data. ThePES packets are divided, and stored in the TS payloads, respectively.When a BD ROM is used, each of the TS packets is given a 4-byteTP_Extra_Header, thus resulting in 192-byte source packets. The sourcepackets are written on the multiplexed data. The TP_Extra_Header storesinformation such as an Arrival_Time_Stamp (ATS). The ATS shows atransfer start time at which each of the TS packets is to be transferredto a PID filter. The source packets are arranged in the multiplexed dataas shown at the bottom of FIG. 49. The numbers incrementing from thehead of the multiplexed data are called source packet numbers (SPNs).

Each of the TS packets included in the multiplexed data includes notonly streams of audio, video, subtitles and others, but also a ProgramAssociation Table (PAT), a Program Map Table (PMT), and a Program ClockReference (PCR). The PAT shows what a PID in a PMT used in themultiplexed data indicates, and a PID of the PAT itself is registered aszero. The PMT stores PIDs of the streams of video, audio, subtitles andothers included in the multiplexed data, and attribute information ofthe streams corresponding to the PIDs. The PMT also has variousdescriptors relating to the multiplexed data. The descriptors haveinformation such as copy control information showing whether copying ofthe multiplexed data is permitted or not. The PCR stores STC timeinformation corresponding to an ATS showing when the PCR packet istransferred to a decoder, in order to achieve synchronization between anArrival Time Clock (ATC) that is a time axis of ATSs, and an System TimeClock (STC) that is a time axis of PTSs and DTSs.

FIG. 50 illustrates the data structure of the PMT in detail. A PMTheader is disposed at the top of the PMT. The PMT header describes thelength of data included in the PMT and others. A plurality ofdescriptors relating to the multiplexed data is disposed after the PMTheader. Information such as the copy control information is described inthe descriptors. After the descriptors, a plurality of pieces of streaminformation relating to the streams included in the multiplexed data isdisposed. Each piece of stream information includes stream descriptorseach describing information, such as a stream type for identifying acompression codec of a stream, a stream PID, and stream attributeinformation (such as a frame rate or an aspect ratio). The streamdescriptors are equal in number to the number of streams in themultiplexed data.

When the multiplexed data is recorded on a recording medium and others,it is recorded together with multiplexed data information files.

Each of the multiplexed data information files is management informationof the multiplexed data as shown in FIG. 51. The multiplexed datainformation files are in one to one correspondence with the multiplexeddata, and each of the files includes multiplexed data information,stream attribute information, and an entry map.

As illustrated in FIG. 51, the multiplexed data information includes asystem rate, a reproduction start time, and a reproduction end time. Thesystem rate indicates the maximum transfer rate at which a system targetdecoder to be described later transfers the multiplexed data to a PIDfilter. The intervals of the ATSs included in the multiplexed data areset to not higher than a system rate. The reproduction start timeindicates a PTS in a video frame at the head of the multiplexed data. Aninterval of one frame is added to a PTS in a video frame at the end ofthe multiplexed data, and the PTS is set to the reproduction end time.

As shown in FIG. 52, a piece of attribute information is registered inthe stream attribute information, for each PID of each stream includedin the multiplexed data. Each piece of attribute information hasdifferent information depending on whether the corresponding stream is avideo stream, an audio stream, a presentation graphics stream, or aninteractive graphics stream. Each piece of video stream attributeinformation carries information including what kind of compression codecis used for compressing the video stream, and the resolution, aspectratio and frame rate of the pieces of picture data that is included inthe video stream. Each piece of audio stream attribute informationcarries information including what kind of compression codec is used forcompressing the audio stream, how many channels are included in theaudio stream, which language the audio stream supports, and how high thesampling frequency is. The video stream attribute information and theaudio stream attribute information are used for initialization of adecoder before the player plays back the information.

In the present embodiment, the multiplexed data to be used is of astream type included in the PMT. Furthermore, when the multiplexed datais recorded on a recording medium, the video stream attributeinformation included in the multiplexed data information is used. Morespecifically, the moving picture coding method or the moving picturecoding apparatus described in each of embodiments includes a step or aunit for allocating unique information indicating video data generatedby the moving picture coding method or the moving picture codingapparatus in each of embodiments, to the stream type included in the PMTor the video stream attribute information. With the configuration, thevideo data generated by the moving picture coding method or the movingpicture coding apparatus described in each of embodiments can bedistinguished from video data that conforms to another standard.

Furthermore, FIG. 53 illustrates steps of the moving picture decodingmethod according to the present embodiment. In Step exS100, the streamtype included in the PMT or the video stream attribute informationincluded in the multiplexed data information is obtained from themultiplexed data. Next, in Step exS101, it is determined whether or notthe stream type or the video stream attribute information indicates thatthe multiplexed data is generated by the moving picture coding method orthe moving picture coding apparatus in each of embodiments. When it isdetermined that the stream type or the video stream attributeinformation indicates that the multiplexed data is generated by themoving picture coding method or the moving picture coding apparatus ineach of embodiments, in Step exS102, decoding is performed by the movingpicture decoding method in each of embodiments. Furthermore, when thestream type or the video stream attribute information indicatesconformance to the conventional standards, such as MPEG-2, MPEG-4 AVC,and VC-1, in Step exS103, decoding is performed by a moving picturedecoding method in conformity with the conventional standards.

As such, allocating a new unique value to the stream type or the videostream attribute information enables determination whether or not themoving picture decoding method or the moving picture decoding apparatusthat is described in each of embodiments can perform decoding. Even whenmultiplexed data that conforms to a different standard is input, anappropriate decoding method or apparatus can be selected. Thus, itbecomes possible to decode information without any error. Furthermore,the moving picture coding method or apparatus, or the moving picturedecoding method or apparatus in the present embodiment can be used inthe devices and systems described above.

Embodiment 11

Each of the moving picture coding method, the moving picture codingapparatus, the moving picture decoding method, and the moving picturedecoding apparatus in each of embodiments is typically achieved in theform of an integrated circuit or a Large Scale Integrated (LSI) circuit.As an example of the LSI, FIG. 54 illustrates a configuration of the LSIex500 that is made into one chip. The LSI ex500 includes elements ex501,ex502, ex503, ex504, ex505, ex506, ex507, ex508, and ex509 to bedescribed below, and the elements are connected to each other through abus ex510. The power supply circuit unit ex505 is activated by supplyingeach of the elements with power when the power supply circuit unit ex505is turned on.

For example, when coding is performed, the LSI ex500 receives an AVsignal from a microphone ex117, a camera ex113, and others through an AVIO ex509 under control of a control unit ex501 including a CPU ex502, amemory controller ex503, a stream controller ex504, and a drivingfrequency control unit ex512. The received AV signal is temporarilystored in an external memory ex511, such as an SDRAM. Under control ofthe control unit ex501, the stored data is segmented into data portionsaccording to the processing amount and speed to be transmitted to asignal processing unit ex507. Then, the signal processing unit ex507codes an audio signal and/or a video signal. Here, the coding of thevideo signal is the coding described in each of embodiments.Furthermore, the signal processing unit ex507 sometimes multiplexes thecoded audio data and the coded video data, and a stream IO ex506provides the multiplexed data outside. The provided multiplexed data istransmitted to the base station ex107, or written on the recordingmedium ex215. When data sets are multiplexed, the data should betemporarily stored in the buffer ex508 so that the data sets aresynchronized with each other.

Although the memory ex511 is an element outside the LSI ex500, it may beincluded in the LSI ex500. The buffer ex508 is not limited to onebuffer, but may be composed of buffers. Furthermore, the LSI ex500 maybe made into one chip or a plurality of chips.

Furthermore, although the control unit ex501 includes the CPU ex502, thememory controller ex503, the stream controller ex504, the drivingfrequency control unit ex512, the configuration of the control unitex501 is not limited to such. For example, the signal processing unitex507 may further include a CPU. Inclusion of another CPU in the signalprocessing unit ex507 can improve the processing speed. Furthermore, asanother example, the CPU ex502 may serve as or be a part of the signalprocessing unit ex507, and, for example, may include an audio signalprocessing unit. In such a case, the control unit ex501 includes thesignal processing unit ex507 or the CPU ex502 including a part of thesignal processing unit ex507.

The name used here is LSI, but it may also be called IC, system LSI,super LSI, or ultra LSI depending on the degree of integration.

Moreover, ways to achieve integration are not limited to the LSI, and aspecial circuit or a general purpose processor and so forth can alsoachieve the integration. Field Programmable Gate Array (FPGA) that canbe programmed after manufacturing LSIs or a reconfigurable processorthat allows re-configuration of the connection or configuration of anLSI can be used for the same purpose.

In the future, with advancement in semiconductor technology, a brand-newtechnology may replace LSI. The functional blocks can be integratedusing such a technology. The possibility is that the present disclosureis applied to biotechnology.

Embodiment 12

When video data generated in the moving picture coding method or by themoving picture coding apparatus described in each of embodiments isdecoded, compared to when video data that conforms to a conventionalstandard, such as MPEG-2, MPEG-4 AVC, and VC-1 is decoded, theprocessing amount probably increases. Thus, the LSI ex500 needs to beset to a driving frequency higher than that of the CPU ex502 to be usedwhen video data in conformity with the conventional standard is decoded.However, when the driving frequency is set higher, there is a problemthat the power consumption increases.

In order to solve the problem, the moving picture decoding apparatus,such as the television ex300 and the LSI ex500 is configured todetermine to which standard the video data conforms, and switch betweenthe driving frequencies according to the determined standard. FIG. 55illustrates a configuration ex800 in the present embodiment. A drivingfrequency switching unit ex803 sets a driving frequency to a higherdriving frequency when video data is generated by the moving picturecoding method or the moving picture coding apparatus described in eachof embodiments. Then, the driving frequency switching unit ex803instructs a decoding processing unit ex801 that executes the movingpicture decoding method described in each of embodiments to decode thevideo data. When the video data conforms to the conventional standard,the driving frequency switching unit ex803 sets a driving frequency to alower driving frequency than that of the video data generated by themoving picture coding method or the moving picture coding apparatusdescribed in each of embodiments. Then, the driving frequency switchingunit ex803 instructs the decoding processing unit ex802 that conforms tothe conventional standard to decode the video data.

More specifically, the driving frequency switching unit ex803 includesthe CPU ex502 and the driving frequency control unit ex512 in FIG. 54.Here, each of the decoding processing unit ex801 that executes themoving picture decoding method described in each of embodiments and thedecoding processing unit ex802 that conforms to the conventionalstandard corresponds to the signal processing unit ex507 in FIG. 54. TheCPU ex502 determines to which standard the video data conforms. Then,the driving frequency control unit ex512 determines a driving frequencybased on a signal from the CPU ex502. Furthermore, the signal processingunit ex507 decodes the video data based on the signal from the CPUex502. For example, the identification information described inEmbodiment 10 is probably used for identifying the video data. Theidentification information is not limited to the one described inEmbodiment 10 but may be any information as long as the informationindicates to which standard the video data conforms. For example, whenwhich standard video data conforms to can be determined based on anexternal signal for determining that the video data is used for atelevision or a disk, etc., the determination may be made based on suchan external signal. Furthermore, the CPU ex502 selects a drivingfrequency based on, for example, a look-up table in which the standardsof the video data are associated with the driving frequencies as shownin FIG. 57. The driving frequency can be selected by storing the look-uptable in the buffer ex508 and in an internal memory of an LSI, and withreference to the look-up table by the CPU ex502.

FIG. 56 illustrates steps for executing a method in the presentembodiment. First, in Step exS200, the signal processing unit ex507obtains identification information from the multiplexed data. Next, inStep exS201, the CPU ex502 determines whether or not the video data isgenerated by the coding method and the coding apparatus described ineach of embodiments, based on the identification information. When thevideo data is generated by the moving picture coding method and themoving picture coding apparatus described in each of embodiments, inStep exS202, the CPU ex502 transmits a signal for setting the drivingfrequency to a higher driving frequency to the driving frequency controlunit ex512. Then, the driving frequency control unit ex512 sets thedriving frequency to the higher driving frequency. On the other hand,when the identification information indicates that the video dataconforms to the conventional standard, such as MPEG-2, MPEG-4 AVC, andVC-1, in Step exS203, the CPU ex502 transmits a signal for setting thedriving frequency to a lower driving frequency to the driving frequencycontrol unit ex512. Then, the driving frequency control unit ex512 setsthe driving frequency to the lower driving frequency than that in thecase where the video data is generated by the moving picture codingmethod and the moving picture coding apparatus described in each ofembodiment.

Furthermore, along with the switching of the driving frequencies, thepower conservation effect can be improved by changing the voltage to beapplied to the LSI ex500 or an apparatus including the LSI ex500. Forexample, when the driving frequency is set lower, the voltage to beapplied to the LSI ex500 or the apparatus including the LSI ex500 isprobably set to a voltage lower than that in the case where the drivingfrequency is set higher.

Furthermore, when the processing amount for decoding is larger, thedriving frequency may be set higher, and when the processing amount fordecoding is smaller, the driving frequency may be set lower as themethod for setting the driving frequency. Thus, the setting method isnot limited to the ones described above. For example, when theprocessing amount for decoding video data in conformity with MPEG-4 AVCis larger than the processing amount for decoding video data generatedby the moving picture coding method and the moving picture codingapparatus described in each of embodiments, the driving frequency isprobably set in reverse order to the setting described above.

Furthermore, the method for setting the driving frequency is not limitedto the method for setting the driving frequency lower. For example, whenthe identification information indicates that the video data isgenerated by the moving picture coding method and the moving picturecoding apparatus described in each of embodiments, the voltage to beapplied to the LSI ex500 or the apparatus including the LSI ex500 isprobably set higher. When the identification information indicates thatthe video data conforms to the conventional standard, such as MPEG-2,MPEG-4 AVC, and VC-1, the voltage to be applied to the LSI ex500 or theapparatus including the LSI ex500 is probably set lower. As anotherexample, when the identification information indicates that the videodata is generated by the moving picture coding method and the movingpicture coding apparatus described in each of embodiments, the drivingof the CPU ex502 does not probably have to be suspended. When theidentification information indicates that the video data conforms to theconventional standard, such as MPEG-2, MPEG-4 AVC, and VC-1, the drivingof the CPU ex502 is probably suspended at a given time because the CPUex502 has extra processing capacity. Even when the identificationinformation indicates that the video data is generated by the movingpicture coding method and the moving picture coding apparatus describedin each of embodiments, in the case where the CPU ex502 has extraprocessing capacity, the driving of the CPU ex502 is probably suspendedat a given time. In such a case, the suspending time is probably setshorter than that in the case where when the identification informationindicates that the video data conforms to the conventional standard,such as MPEG-2, MPEG-4 AVC, and VC-1.

Accordingly, the power conservation effect can be improved by switchingbetween the driving frequencies in accordance with the standard to whichthe video data conforms. Furthermore, when the LSI ex500 or theapparatus including the LSI ex500 is driven using a battery, the batterylife can be extended with the power conservation effect.

Embodiment 13

There are cases where a plurality of video data that conforms todifferent standards, is provided to the devices and systems, such as atelevision and a cellular phone. In order to enable decoding theplurality of video data that conforms to the different standards, thesignal processing unit ex507 of the LSI ex500 needs to conform to thedifferent standards. However, the problems of increase in the scale ofthe circuit of the LSI ex500 and increase in the cost arise with theindividual use of the signal processing units ex507 that conform to therespective standards.

In order to solve the problem, what is conceived is a configuration inwhich the decoding processing unit for implementing the moving picturedecoding method described in each of embodiments and the decodingprocessing unit that conforms to the conventional standard, such asMPEG-2, MPEG-4 AVC, and VC-1 are partly shared. Ex900 in FIG. 58A showsan example of the configuration. For example, the moving picturedecoding method described in each of embodiments and the moving picturedecoding method that conforms to MPEG-4 AVC have, partly in common, thedetails of processing, such as entropy coding, inverse quantization,deblocking filtering, and motion compensated prediction. The details ofprocessing to be shared probably include use of a decoding processingunit ex902 that conforms to MPEG-4 AVC. In contrast, a dedicateddecoding processing unit ex901 is probably used for other processingunique to an aspect of the present disclosure. Since the aspect of thepresent disclosure is characterized by inverse quantization inparticular, for example, the dedicated decoding processing unit ex901 isused for inverse quantization. Otherwise, the decoding processing unitis probably shared for one of the entropy decoding, deblockingfiltering, and motion compensation, or all of the processing. Thedecoding processing unit for implementing the moving picture decodingmethod described in each of embodiments may be shared for the processingto be shared, and a dedicated decoding processing unit may be used forprocessing unique to that of MPEG-4 AVC.

Furthermore, ex1000 in FIG. 58B shows another example in that processingis partly shared. This example uses a configuration including adedicated decoding processing unit ex1001 that supports the processingunique to an aspect of the present disclosure, a dedicated decodingprocessing unit ex1002 that supports the processing unique to anotherconventional standard, and a decoding processing unit ex1003 thatsupports processing to be shared between the moving picture decodingmethod according to the aspect of the present disclosure and theconventional moving picture decoding method. Here, the dedicateddecoding processing units ex1001 and ex1002 are not necessarilyspecialized for the processing according to the aspect of the presentdisclosure and the processing of the conventional standard,respectively, and may be the ones capable of implementing generalprocessing. Furthermore, the configuration of the present embodiment canbe implemented by the LSI ex500.

As such, reducing the scale of the circuit of an LSI and reducing thecost are possible by sharing the decoding processing unit for theprocessing to be shared between the moving picture decoding methodaccording to the aspect of the present disclosure and the moving picturedecoding method in conformity with the conventional standard.

INDUSTRIAL APPLICABILITY

The image coding method and image decoding method according to an aspectof the present disclosure is advantageously applicable to a movingpicture coding method and a moving picture decoding method.

1. An image coding method for coding an image on a block-by-block basisto generate a bitstream, the method comprising: deriving, as a firstmerging candidate, a merging candidate based on a prediction direction,a motion vector, and a reference picture index which have been used forcoding a block spatially or temporally neighboring a current block to becoded, the merging candidate being a combination of a predictiondirection, a motion vector, and a reference picture index for use incoding of the current block; deriving, as a second merging candidate, amerging candidate having a motion vector which is a predeterminedvector; selecting a merging candidate to be used for the coding of thecurrent block from the derived first merging candidate and the derivedsecond merging candidate; and attaching an index for identifying theselected merging candidate to the bitstream.
 2. The image coding methodaccording to claim 1, wherein in the deriving of a merging candidate asa second merging candidate, the second merging candidate is derived foreach referable reference picture.
 3. The image coding method accordingto claim 1, wherein the predetermined vector is a zero vector.
 4. Theimage coding method according to claim 1, further comprising:determining a maximum number of merging candidates; and determiningwhether or not the total number of the derived first merging candidateis smaller than the maximum number, wherein the deriving of a mergingcandidate as a second merging candidate is performed when it isdetermined that the total number of the first merging candidate issmaller than the maximum number.
 5. The image coding method according toclaim 4, wherein in the attaching, the index is coded using thedetermined maximum number, and the coded index is attached to thebitstream.
 6. The image coding method according to claim 4, wherein inthe attaching, information indicating the determined maximum number isfurther attached to the bitstream.
 7. The image coding method accordingto claim 1, wherein in the deriving of a merging candidate as a firstmerging candidate, a merging candidate which is a combination of aprediction direction, a motion vector, and a reference picture index isderived as the first merging candidate, the combination being differentfrom a combination of a prediction direction, a motion vector, and areference picture index of any first merging candidate previouslyderived.
 8. The image coding method according to claim 1, furthercomprising: switching a coding process between a first coding processconforming to a first standard and a second coding process conforming toa second standard; and attaching, to the bitstream, identificationinformation indicating either the first standard or the second standardto which the coding process after the switching conforms, wherein whenthe coding process is switched to the first coding process, the derivingof a merging candidate as a first merging candidate, the deriving of amerging candidate as a second merging candidate, the selecting, and theattaching are performed as the first coding process.
 9. An image codingapparatus which codes an image on a block-by-block basis to generate abitstream, the apparatus comprising: a first derivation unit configuredto derive, as a first merging candidate, a merging candidate based on aprediction direction, a motion vector, and a reference picture indexwhich have been used for coding a block spatially or temporallyneighboring a current block to be coded, the merging candidate being acombination of a prediction direction, a motion vector, and a referencepicture index for use in coding of the current block; a secondderivation unit configured to derive, as a second merging candidate, amerging candidate having a motion vector which is a predeterminedvector; a prediction control unit configured to select a mergingcandidate to be used for the coding of the current block from thederived first merging candidate and the derived second mergingcandidate; and a coding unit configured to attach an index foridentifying the selected merging candidate to the bitstream.
 10. Animage decoding method for decoding, on a block-by-block basis, a codedimage included in a bitstream, the method comprising: deriving, as afirst merging candidate, a merging candidate based on a predictiondirection, a motion vector, and a reference picture index which havebeen used for decoding a block spatially or temporally neighboring acurrent block to be decoded, the merging candidate being a combinationof a prediction direction, a motion vector, and a reference pictureindex for use in decoding of the current block; deriving, as a secondmerging candidate, a merging candidate having a motion vector which is apredetermined vector; obtaining, from the bitstream, an index foridentifying a merging candidate; and selecting, based on the obtainedindex, a merging candidate from the first merging candidate and thesecond merging candidate, the merging candidate being to be used for thedecoding of a current block.
 11. The image decoding method according toclaim 10, wherein in the deriving of a merging candidate as a secondmerging candidate, the second merging candidate is derived for eachreferable reference picture.
 12. The image decoding method according toclaim 10, wherein the predetermined vector is a zero vector.
 13. Theimage decoding method according to claim 10, further comprising:determining a maximum number of merging candidates; and determiningwhether or not the total number of the derived first merging candidateis smaller than the maximum number, wherein the deriving of a mergingcandidate as a second merging candidate is performed when it isdetermined that the total number of the derived first merging candidateis smaller than the maximum number.
 14. The image decoding methodaccording to claim 13, wherein in the obtaining, the index coded andattached to the bitstream is obtained by decoding the index using thedetermined maximum number.
 15. The image decoding method according toclaim 13, wherein in the determining of a maximum number of mergingcandidates, the maximum number is determined based on informationattached to the bitstream and indicating the maximum number.
 16. Theimage decoding method according to claim 10, wherein in the deriving ofa merging candidate as a first merging candidate, a merging candidatewhich is a combination of a prediction direction, a motion vector, and areference picture index is derived as the first merging candidate, thecombination being different from a combination of a predictiondirection, a motion vector, and a reference picture index of any firstmerging candidate previously derived.
 17. The image decoding methodaccording to claim 10, further comprising: switching a decoding processbetween a first decoding process conforming to a first standard and asecond decoding process conforming to a second standard, according toidentification information which is attached to the bitstream andindicates either the first standard or the second standard, wherein whenthe decoding process is switched to the first decoding process, thederiving of a merging candidate as a first merging candidate, thederiving of a merging candidate as a second merging candidate, theobtaining, and the selecting are performed as the first decodingprocess.
 18. An image decoding apparatus which decodes, on ablock-by-block basis, a coded image included in a bitstream, theapparatus comprising: a first derivation unit configured to derive, as afirst merging candidate, a merging candidate based on a predictiondirection, a motion vector, and a reference picture index which havebeen used for decoding a block spatially or temporally neighboring acurrent block to be decoded, the merging candidate being a combinationof a prediction direction, a motion vector, and a reference pictureindex for use in decoding of the current block; a second derivation unitconfigured to derive, as a second merging candidate, a merging candidatehaving a motion vector which is a predetermined vector; a decoding unitconfigured to obtain, from the bitstream, an index for identifying amerging candidate; and a prediction control unit configured to select,based on the obtained index, a merging candidate from the first mergingcandidate and the second merging candidate, the merging candidate beingto be used for the decoding of a current block.
 19. An image coding anddecoding apparatus comprising: an image coding apparatus an image on ablock-by-block basis to generate a bitstream, the apparatus comprising:a first derivation unit configured to derive, as a first mergingcandidate, a merging candidate based on a prediction direction, a motionvector, and a reference picture index which have been used for coding ablock spatially or temporally neighboring a current block to be coded,the merging candidate being a combination of a prediction direction, amotion vector, and a reference picture index for use in coding of thecurrent block; a second derivation unit configured to derive, as asecond merging candidate, a merging candidate having a motion vectorwhich is a predetermined vector; a prediction control unit configured toselect a merging candidate to be used for the coding of the currentblock from the derived first merging candidate and the derived secondmerging candidate; and a coding unit configured to attach an index foridentifying the selected merging candidate to the bitstream; anddecoding apparatus according to claim 18.